|
--- |
|
license: apache-2.0 |
|
base_model: hZzy/qwen2.5-0.5b-sft-news-IFT |
|
tags: |
|
- alignment-handbook |
|
- ndcg |
|
- trl |
|
- expo |
|
- generated_from_trainer |
|
- trl |
|
- expo |
|
- generated_from_trainer |
|
datasets: |
|
- hZzy/train_pairwise_weighted |
|
model-index: |
|
- name: qwen2.5-0.5b-expo-L2EXPO-W0-noES2-0.1 |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/zhiyuzha-university-of-florida/huggingface/runs/t7vhabyt) |
|
# qwen2.5-0.5b-expo-L2EXPO-W0-noES2-0.1 |
|
|
|
This model is a fine-tuned version of [hZzy/qwen2.5-0.5b-sft-news-IFT](https://huggingface.co/hZzy/qwen2.5-0.5b-sft-news-IFT) on the hZzy/train_pairwise_weighted dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 187.9473 |
|
- Logps: -88.4161 |
|
- Logits: -1.2945 |
|
- Objective: 183.8510 |
|
- Dpo Loss: 0.6799 |
|
- Regularize: 0.4168 |
|
- Ranking Simple: 0.5326 |
|
- Ranking Idealized: 0.6025 |
|
- Ranking Idealized Expo: 0.5233 |
|
- Wo Beta: 15.9953 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 1e-06 |
|
- train_batch_size: 4 |
|
- eval_batch_size: 4 |
|
- seed: 42 |
|
- distributed_type: multi-GPU |
|
- num_devices: 3 |
|
- gradient_accumulation_steps: 12 |
|
- total_train_batch_size: 144 |
|
- total_eval_batch_size: 12 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: cosine |
|
- lr_scheduler_warmup_ratio: 0.1 |
|
- num_epochs: 5 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Logps | Logits | Objective | Dpo Loss | Regularize | Ranking Simple | Ranking Idealized | Ranking Idealized Expo | Wo Beta | |
|
|:-------------:|:------:|:----:|:---------------:|:--------:|:-------:|:---------:|:--------:|:----------:|:--------------:|:-----------------:|:----------------------:|:-------:| |
|
| 181.5404 | 0.1417 | 50 | 182.3598 | -90.9189 | -1.4233 | 180.3279 | 0.6890 | 0.4088 | 0.5264 | 0.6025 | 0.5233 | 16.2974 | |
|
| 156.9096 | 0.2834 | 100 | 181.8641 | -91.4510 | -1.4702 | 180.3150 | 0.6855 | 0.4101 | 0.5316 | 0.6025 | 0.5233 | 16.3731 | |
|
| 145.838 | 0.4251 | 150 | 180.6479 | -90.7049 | -1.4504 | 178.1705 | 0.6790 | 0.4023 | 0.5383 | 0.6025 | 0.5233 | 16.5880 | |
|
| 140.9398 | 0.5668 | 200 | 184.4987 | -90.3330 | -1.3895 | 181.5451 | 0.6803 | 0.4101 | 0.5326 | 0.6025 | 0.5233 | 16.1415 | |
|
| 131.1439 | 0.7085 | 250 | 182.2077 | -91.2246 | -1.4789 | 178.4409 | 0.6797 | 0.4059 | 0.5326 | 0.6025 | 0.5233 | 16.3683 | |
|
| 118.3192 | 0.8503 | 300 | 183.4459 | -92.5771 | -1.4552 | 180.4714 | 0.6817 | 0.4123 | 0.5326 | 0.6025 | 0.5233 | 16.4041 | |
|
| 108.5029 | 0.9920 | 350 | 183.9593 | -92.1804 | -1.4317 | 180.1151 | 0.6782 | 0.4095 | 0.5321 | 0.6025 | 0.5233 | 16.3367 | |
|
| 104.6813 | 1.1337 | 400 | 183.8759 | -89.7261 | -1.3930 | 180.2840 | 0.6801 | 0.4094 | 0.5311 | 0.6025 | 0.5233 | 16.2209 | |
|
| 90.5585 | 1.2754 | 450 | 184.0673 | -91.2037 | -1.3663 | 180.6296 | 0.6795 | 0.4105 | 0.5357 | 0.6025 | 0.5233 | 16.2889 | |
|
| 91.2372 | 1.4171 | 500 | 185.7194 | -89.4298 | -1.3281 | 180.8790 | 0.6782 | 0.4110 | 0.5347 | 0.6025 | 0.5233 | 16.0443 | |
|
| 85.7307 | 1.5588 | 550 | 186.2241 | -91.6866 | -1.3683 | 182.1382 | 0.6799 | 0.4147 | 0.5336 | 0.6025 | 0.5233 | 16.1863 | |
|
| 79.9458 | 1.7005 | 600 | 186.2137 | -91.0847 | -1.3519 | 181.8686 | 0.6794 | 0.4135 | 0.5373 | 0.6025 | 0.5233 | 16.1060 | |
|
| 86.7578 | 1.8422 | 650 | 186.7196 | -89.4070 | -1.3403 | 182.4970 | 0.6797 | 0.4141 | 0.5316 | 0.6025 | 0.5233 | 16.0270 | |
|
| 76.2665 | 1.9839 | 700 | 186.2802 | -89.5857 | -1.3223 | 182.3933 | 0.6800 | 0.4136 | 0.5311 | 0.6025 | 0.5233 | 16.1117 | |
|
| 65.1575 | 2.1256 | 750 | 188.1571 | -90.2454 | -1.3253 | 184.2076 | 0.6806 | 0.4179 | 0.5321 | 0.6025 | 0.5233 | 15.9179 | |
|
| 66.0375 | 2.2674 | 800 | 186.7221 | -88.5874 | -1.3137 | 181.9355 | 0.6781 | 0.4137 | 0.5336 | 0.6025 | 0.5233 | 15.9879 | |
|
| 55.6773 | 2.4091 | 850 | 189.5397 | -88.2689 | -1.3112 | 185.2096 | 0.6809 | 0.4203 | 0.5300 | 0.6025 | 0.5233 | 15.9311 | |
|
| 54.3682 | 2.5508 | 900 | 188.2381 | -88.3611 | -1.3259 | 184.1678 | 0.6793 | 0.4167 | 0.5311 | 0.6025 | 0.5233 | 15.9680 | |
|
| 50.3775 | 2.6925 | 950 | 189.5419 | -88.8005 | -1.3091 | 185.0044 | 0.6802 | 0.4183 | 0.5331 | 0.6025 | 0.5233 | 15.9986 | |
|
| 45.9449 | 2.8342 | 1000 | 187.7079 | -87.8148 | -1.2990 | 183.5676 | 0.6792 | 0.4161 | 0.5300 | 0.6025 | 0.5233 | 15.9960 | |
|
| 49.0003 | 2.9759 | 1050 | 188.0040 | -88.3004 | -1.2778 | 184.0016 | 0.6792 | 0.4173 | 0.5342 | 0.6025 | 0.5233 | 16.0403 | |
|
| 40.2428 | 3.1176 | 1100 | 188.7166 | -88.6470 | -1.2988 | 184.3815 | 0.6801 | 0.4181 | 0.5326 | 0.6025 | 0.5233 | 15.9981 | |
|
| 37.177 | 3.2593 | 1150 | 188.2563 | -87.9239 | -1.2843 | 184.3351 | 0.6804 | 0.4183 | 0.5357 | 0.6025 | 0.5233 | 16.0123 | |
|
| 34.9809 | 3.4010 | 1200 | 189.1705 | -88.1129 | -1.2900 | 184.8718 | 0.6806 | 0.4193 | 0.5326 | 0.6025 | 0.5233 | 15.9531 | |
|
| 34.073 | 3.5427 | 1250 | 188.2203 | -88.5617 | -1.2892 | 184.1966 | 0.6802 | 0.4177 | 0.5336 | 0.6025 | 0.5233 | 16.0022 | |
|
| 28.4565 | 3.6845 | 1300 | 188.3189 | -88.0836 | -1.2942 | 184.1293 | 0.6803 | 0.4178 | 0.5331 | 0.6025 | 0.5233 | 16.0081 | |
|
| 27.4636 | 3.8262 | 1350 | 188.3022 | -88.4586 | -1.2973 | 184.2191 | 0.6803 | 0.4178 | 0.5321 | 0.6025 | 0.5233 | 15.9996 | |
|
| 27.3902 | 3.9679 | 1400 | 187.9691 | -88.3135 | -1.2974 | 183.7816 | 0.6798 | 0.4168 | 0.5321 | 0.6025 | 0.5233 | 15.9788 | |
|
| 21.2906 | 4.1096 | 1450 | 187.8985 | -88.1212 | -1.2976 | 183.6546 | 0.6796 | 0.4164 | 0.5321 | 0.6025 | 0.5233 | 15.9853 | |
|
| 19.8787 | 4.2513 | 1500 | 188.0825 | -88.3078 | -1.2942 | 183.8684 | 0.6799 | 0.4169 | 0.5321 | 0.6025 | 0.5233 | 15.9839 | |
|
| 18.4741 | 4.3930 | 1550 | 188.0407 | -88.4855 | -1.2951 | 184.0446 | 0.6802 | 0.4173 | 0.5326 | 0.6025 | 0.5233 | 15.9950 | |
|
| 20.4794 | 4.5347 | 1600 | 187.9061 | -88.4381 | -1.2950 | 183.8276 | 0.6799 | 0.4168 | 0.5331 | 0.6025 | 0.5233 | 16.0004 | |
|
| 17.2115 | 4.6764 | 1650 | 187.9504 | -88.4174 | -1.2938 | 183.8566 | 0.6798 | 0.4168 | 0.5326 | 0.6025 | 0.5233 | 15.9942 | |
|
| 16.5799 | 4.8181 | 1700 | 187.9360 | -88.4220 | -1.2946 | 183.8405 | 0.6799 | 0.4168 | 0.5326 | 0.6025 | 0.5233 | 15.9963 | |
|
| 16.689 | 4.9598 | 1750 | 187.9473 | -88.4162 | -1.2945 | 183.8510 | 0.6799 | 0.4168 | 0.5326 | 0.6025 | 0.5233 | 15.9953 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.42.0 |
|
- Pytorch 2.3.0+cu121 |
|
- Datasets 3.2.0 |
|
- Tokenizers 0.19.1 |
|
|
