Hugging Face's logo

hZzy
/
qwen2.5-0.5b-expo-L2EXPO-EXPERIMENT-0.1-5e6

Safetensors
qwen2
alignment-handbook
ndcg
trl
expo
Generated from Trainer
Model card Files Community
qwen2.5-0.5b-expo-L2EXPO-EXPERIMENT-0.1-5e6 / README.md
hZzy's picture
hZzy
End of training
99837b1 verified
preview code
|
raw
history blame contribute delete
5.18 kB
metadata
license: apache-2.0
base_model: hZzy/qwen2.5-0.5b-sft-news-IFT
tags:
  - alignment-handbook
  - ndcg
  - trl
  - expo
  - generated_from_trainer
  - trl
  - expo
  - generated_from_trainer
datasets:
  - hZzy/train_pairwise
model-index:
  - name: qwen2.5-0.5b-expo-L2EXPO-EXPERIMENT-0.1-5e6
    results: []

Visualize in Weights & Biases

qwen2.5-0.5b-expo-L2EXPO-EXPERIMENT-0.1-5e6

This model is a fine-tuned version of hZzy/qwen2.5-0.5b-sft-news-IFT on the hZzy/train_pairwise dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5835
  • Logps: -80.5388
  • Logits: -0.6811
  • Objective: 0.5760
  • Dpo Loss: 0.7150
  • Regularize: 0.5760
  • Ranking Simple: 0.5248
  • Ranking Idealized: 0.5888
  • Ranking Idealized Expo: 0.5103

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-06
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 6
  • gradient_accumulation_steps: 12
  • total_train_batch_size: 288
  • total_eval_batch_size: 24
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss Logps Logits Objective Dpo Loss Regularize Ranking Simple Ranking Idealized Ranking Idealized Expo
0.3559 0.2834 50 0.4230 -96.0254 -1.4739 0.4318 0.6921 0.4318 0.5186 0.5888 0.5103
0.3834 0.5668 100 0.4808 -85.0174 -1.1784 0.4771 0.6952 0.4771 0.5155 0.5888 0.5103
0.3746 0.8503 150 0.5245 -81.6313 -1.0148 0.5243 0.7075 0.5243 0.5165 0.5888 0.5103
0.3365 1.1337 200 0.5510 -80.3085 -1.0812 0.5435 0.7045 0.5435 0.5134 0.5888 0.5103
0.2986 1.4171 250 0.5600 -79.8608 -0.9740 0.5590 0.7114 0.5590 0.5217 0.5888 0.5103
0.2571 1.7005 300 0.5774 -77.6594 -0.8023 0.5724 0.7149 0.5724 0.5217 0.5888 0.5103
0.2355 1.9839 350 0.5797 -79.4555 -0.7278 0.5736 0.7176 0.5736 0.5186 0.5888 0.5103
0.1974 2.2674 400 0.5802 -81.3670 -0.7596 0.5785 0.7156 0.5785 0.5279 0.5888 0.5103
0.1787 2.5508 450 0.5830 -80.8003 -0.7106 0.5799 0.7161 0.5799 0.5227 0.5888 0.5103
0.1582 2.8342 500 0.5836 -80.3096 -0.7272 0.5800 0.7177 0.5800 0.5176 0.5888 0.5103
0.1257 3.1176 550 0.5853 -80.8767 -0.6681 0.5816 0.7178 0.5816 0.5238 0.5888 0.5103
0.1018 3.4010 600 0.5870 -80.2631 -0.6520 0.5793 0.7155 0.5793 0.5227 0.5888 0.5103
0.0908 3.6845 650 0.5846 -80.0938 -0.6950 0.5751 0.7142 0.5751 0.5310 0.5888 0.5103
0.0782 3.9679 700 0.5832 -80.5798 -0.6694 0.5775 0.7158 0.5775 0.5217 0.5888 0.5103
0.0573 4.2513 750 0.5847 -80.5919 -0.6764 0.5775 0.7156 0.5775 0.5238 0.5888 0.5103
0.0513 4.5347 800 0.5835 -80.5038 -0.6806 0.5758 0.7149 0.5758 0.5248 0.5888 0.5103
0.0447 4.8181 850 0.5835 -80.5460 -0.6807 0.5761 0.7150 0.5761 0.5248 0.5888 0.5103

Framework versions

  • Transformers 4.42.0
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1