Hugging Face's logo

hZzy
/
qwen2.5-0.5b-expo-DPO-ES-0.01

Safetensors
qwen2
alignment-handbook
ndcg
trl
expo
Generated from Trainer
Model card Files Community
qwen2.5-0.5b-expo-DPO-ES-0.01 / README.md
hZzy's picture
hZzy
End of training
b1acf0c verified
preview code
|
raw
history blame
3.37 kB
metadata
license: apache-2.0
base_model: hZzy/qwen2.5-0.5b-sft-news-IFT
tags:
  - alignment-handbook
  - ndcg
  - trl
  - expo
  - generated_from_trainer
  - trl
  - expo
  - generated_from_trainer
datasets:
  - hZzy/train_pairwise
model-index:
  - name: qwen2.5-0.5b-expo-DPO-ES-0.01
    results: []

Visualize in Weights & Biases

qwen2.5-0.5b-expo-DPO-ES-0.01

This model is a fine-tuned version of hZzy/qwen2.5-0.5b-sft-news-IFT on the hZzy/train_pairwise dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6894
  • Logps: -99.3821
  • Logits: -1.7611
  • Objective: 0.6902
  • Dpo Loss: 0.6902
  • Regularize: 0.6902
  • Ranking Simple: 0.5305
  • Ranking Idealized: 0.8732
  • Ranking Idealized Expo: 0.5321
  • Wo Beta: 9.3575

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-06
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 3
  • gradient_accumulation_steps: 12
  • total_train_batch_size: 144
  • total_eval_batch_size: 12
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 5

Training results

Training Loss Epoch Step Validation Loss Logps Logits Objective Dpo Loss Regularize Ranking Simple Ranking Idealized Ranking Idealized Expo Wo Beta
0.6907 0.1417 50 0.6894 -99.3821 -1.7611 0.6902 0.6902 0.6902 0.5305 0.8732 0.5321 9.3575
0.6701 0.2834 100 0.6837 -153.5716 -1.8467 0.6896 0.6896 0.6896 0.5518 0.8732 0.5321 14.1741
0.637 0.4251 150 0.6723 -198.9269 -2.4614 0.6765 0.6765 0.6765 0.5823 0.8732 0.5321 17.2098
0.5833 0.5668 200 0.6729 -256.0312 -3.3478 0.6780 0.6780 0.6780 0.5797 0.8732 0.5321 21.7109
0.5439 0.7085 250 0.6781 -257.3546 -3.6269 0.6858 0.6858 0.6858 0.5683 0.8732 0.5321 22.9139
0.5077 0.8503 300 0.6640 -319.3935 -4.6100 0.6685 0.6685 0.6685 0.5828 0.8732 0.5321 23.6506

Framework versions

  • Transformers 4.42.0
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1