Hugging Face's logo

RyanYr
/
bt-rm-llama3.1-hendrydong-preference_700K

Text Classification
Transformers
Safetensors
llama
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
Edit model card

bt-rm

This model was trained from LLaMA 3.1 8B Instruct with dataset hendrydong/preference_700K (Preprocessed dataset RyanYr/preference_700K_llama31_tokenized). Training script is https://github.com/yurun-yuan/RLHF-Reward-Modeling/blob/4b827117dc9a85062c396eb62200b48e6dbfd596/bradley-terry-rm/llama3_rm.py

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-06
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 4
  • gradient_accumulation_steps: 256
  • total_train_batch_size: 1024
  • total_eval_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.03
  • num_epochs: 1

Training results

Framework versions

  • Transformers 4.43.3
  • Pytorch 2.1.2+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1
Downloads last month
39
Safetensors
Model size
7.5B params
Tensor type
BF16
·
Inference Examples
Text Classification
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Collection including RyanYr/bt-rm-llama3.1-hendrydong-preference_700K

Evaluation results

Metadata error: specify a dataset to view leaderboard