File size: 8,653 Bytes
29fb157 0674aa8 29fb157 0674aa8 29fb157 0674aa8 29fb157 0674aa8 29fb157 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 |
---
license: apache-2.0
base_model: hZzy/qwen2.5-0.5b-sft-news-IFT
tags:
- alignment-handbook
- ndcg
- trl
- expo
- generated_from_trainer
- trl
- expo
- generated_from_trainer
datasets:
- hZzy/train_pairwise_weighted
model-index:
- name: qwen2.5-0.5b-expo-L2EXPO-W0-noES2-0.1
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/zhiyuzha-university-of-florida/huggingface/runs/t7vhabyt)
# qwen2.5-0.5b-expo-L2EXPO-W0-noES2-0.1
This model is a fine-tuned version of [hZzy/qwen2.5-0.5b-sft-news-IFT](https://huggingface.co/hZzy/qwen2.5-0.5b-sft-news-IFT) on the hZzy/train_pairwise_weighted dataset.
It achieves the following results on the evaluation set:
- Loss: 187.9473
- Logps: -88.4161
- Logits: -1.2945
- Objective: 183.8510
- Dpo Loss: 0.6799
- Regularize: 0.4168
- Ranking Simple: 0.5326
- Ranking Idealized: 0.6025
- Ranking Idealized Expo: 0.5233
- Wo Beta: 15.9953
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-06
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 3
- gradient_accumulation_steps: 12
- total_train_batch_size: 144
- total_eval_batch_size: 12
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 5
### Training results
| Training Loss | Epoch | Step | Validation Loss | Logps | Logits | Objective | Dpo Loss | Regularize | Ranking Simple | Ranking Idealized | Ranking Idealized Expo | Wo Beta |
|:-------------:|:------:|:----:|:---------------:|:--------:|:-------:|:---------:|:--------:|:----------:|:--------------:|:-----------------:|:----------------------:|:-------:|
| 181.5404 | 0.1417 | 50 | 182.3598 | -90.9189 | -1.4233 | 180.3279 | 0.6890 | 0.4088 | 0.5264 | 0.6025 | 0.5233 | 16.2974 |
| 156.9096 | 0.2834 | 100 | 181.8641 | -91.4510 | -1.4702 | 180.3150 | 0.6855 | 0.4101 | 0.5316 | 0.6025 | 0.5233 | 16.3731 |
| 145.838 | 0.4251 | 150 | 180.6479 | -90.7049 | -1.4504 | 178.1705 | 0.6790 | 0.4023 | 0.5383 | 0.6025 | 0.5233 | 16.5880 |
| 140.9398 | 0.5668 | 200 | 184.4987 | -90.3330 | -1.3895 | 181.5451 | 0.6803 | 0.4101 | 0.5326 | 0.6025 | 0.5233 | 16.1415 |
| 131.1439 | 0.7085 | 250 | 182.2077 | -91.2246 | -1.4789 | 178.4409 | 0.6797 | 0.4059 | 0.5326 | 0.6025 | 0.5233 | 16.3683 |
| 118.3192 | 0.8503 | 300 | 183.4459 | -92.5771 | -1.4552 | 180.4714 | 0.6817 | 0.4123 | 0.5326 | 0.6025 | 0.5233 | 16.4041 |
| 108.5029 | 0.9920 | 350 | 183.9593 | -92.1804 | -1.4317 | 180.1151 | 0.6782 | 0.4095 | 0.5321 | 0.6025 | 0.5233 | 16.3367 |
| 104.6813 | 1.1337 | 400 | 183.8759 | -89.7261 | -1.3930 | 180.2840 | 0.6801 | 0.4094 | 0.5311 | 0.6025 | 0.5233 | 16.2209 |
| 90.5585 | 1.2754 | 450 | 184.0673 | -91.2037 | -1.3663 | 180.6296 | 0.6795 | 0.4105 | 0.5357 | 0.6025 | 0.5233 | 16.2889 |
| 91.2372 | 1.4171 | 500 | 185.7194 | -89.4298 | -1.3281 | 180.8790 | 0.6782 | 0.4110 | 0.5347 | 0.6025 | 0.5233 | 16.0443 |
| 85.7307 | 1.5588 | 550 | 186.2241 | -91.6866 | -1.3683 | 182.1382 | 0.6799 | 0.4147 | 0.5336 | 0.6025 | 0.5233 | 16.1863 |
| 79.9458 | 1.7005 | 600 | 186.2137 | -91.0847 | -1.3519 | 181.8686 | 0.6794 | 0.4135 | 0.5373 | 0.6025 | 0.5233 | 16.1060 |
| 86.7578 | 1.8422 | 650 | 186.7196 | -89.4070 | -1.3403 | 182.4970 | 0.6797 | 0.4141 | 0.5316 | 0.6025 | 0.5233 | 16.0270 |
| 76.2665 | 1.9839 | 700 | 186.2802 | -89.5857 | -1.3223 | 182.3933 | 0.6800 | 0.4136 | 0.5311 | 0.6025 | 0.5233 | 16.1117 |
| 65.1575 | 2.1256 | 750 | 188.1571 | -90.2454 | -1.3253 | 184.2076 | 0.6806 | 0.4179 | 0.5321 | 0.6025 | 0.5233 | 15.9179 |
| 66.0375 | 2.2674 | 800 | 186.7221 | -88.5874 | -1.3137 | 181.9355 | 0.6781 | 0.4137 | 0.5336 | 0.6025 | 0.5233 | 15.9879 |
| 55.6773 | 2.4091 | 850 | 189.5397 | -88.2689 | -1.3112 | 185.2096 | 0.6809 | 0.4203 | 0.5300 | 0.6025 | 0.5233 | 15.9311 |
| 54.3682 | 2.5508 | 900 | 188.2381 | -88.3611 | -1.3259 | 184.1678 | 0.6793 | 0.4167 | 0.5311 | 0.6025 | 0.5233 | 15.9680 |
| 50.3775 | 2.6925 | 950 | 189.5419 | -88.8005 | -1.3091 | 185.0044 | 0.6802 | 0.4183 | 0.5331 | 0.6025 | 0.5233 | 15.9986 |
| 45.9449 | 2.8342 | 1000 | 187.7079 | -87.8148 | -1.2990 | 183.5676 | 0.6792 | 0.4161 | 0.5300 | 0.6025 | 0.5233 | 15.9960 |
| 49.0003 | 2.9759 | 1050 | 188.0040 | -88.3004 | -1.2778 | 184.0016 | 0.6792 | 0.4173 | 0.5342 | 0.6025 | 0.5233 | 16.0403 |
| 40.2428 | 3.1176 | 1100 | 188.7166 | -88.6470 | -1.2988 | 184.3815 | 0.6801 | 0.4181 | 0.5326 | 0.6025 | 0.5233 | 15.9981 |
| 37.177 | 3.2593 | 1150 | 188.2563 | -87.9239 | -1.2843 | 184.3351 | 0.6804 | 0.4183 | 0.5357 | 0.6025 | 0.5233 | 16.0123 |
| 34.9809 | 3.4010 | 1200 | 189.1705 | -88.1129 | -1.2900 | 184.8718 | 0.6806 | 0.4193 | 0.5326 | 0.6025 | 0.5233 | 15.9531 |
| 34.073 | 3.5427 | 1250 | 188.2203 | -88.5617 | -1.2892 | 184.1966 | 0.6802 | 0.4177 | 0.5336 | 0.6025 | 0.5233 | 16.0022 |
| 28.4565 | 3.6845 | 1300 | 188.3189 | -88.0836 | -1.2942 | 184.1293 | 0.6803 | 0.4178 | 0.5331 | 0.6025 | 0.5233 | 16.0081 |
| 27.4636 | 3.8262 | 1350 | 188.3022 | -88.4586 | -1.2973 | 184.2191 | 0.6803 | 0.4178 | 0.5321 | 0.6025 | 0.5233 | 15.9996 |
| 27.3902 | 3.9679 | 1400 | 187.9691 | -88.3135 | -1.2974 | 183.7816 | 0.6798 | 0.4168 | 0.5321 | 0.6025 | 0.5233 | 15.9788 |
| 21.2906 | 4.1096 | 1450 | 187.8985 | -88.1212 | -1.2976 | 183.6546 | 0.6796 | 0.4164 | 0.5321 | 0.6025 | 0.5233 | 15.9853 |
| 19.8787 | 4.2513 | 1500 | 188.0825 | -88.3078 | -1.2942 | 183.8684 | 0.6799 | 0.4169 | 0.5321 | 0.6025 | 0.5233 | 15.9839 |
| 18.4741 | 4.3930 | 1550 | 188.0407 | -88.4855 | -1.2951 | 184.0446 | 0.6802 | 0.4173 | 0.5326 | 0.6025 | 0.5233 | 15.9950 |
| 20.4794 | 4.5347 | 1600 | 187.9061 | -88.4381 | -1.2950 | 183.8276 | 0.6799 | 0.4168 | 0.5331 | 0.6025 | 0.5233 | 16.0004 |
| 17.2115 | 4.6764 | 1650 | 187.9504 | -88.4174 | -1.2938 | 183.8566 | 0.6798 | 0.4168 | 0.5326 | 0.6025 | 0.5233 | 15.9942 |
| 16.5799 | 4.8181 | 1700 | 187.9360 | -88.4220 | -1.2946 | 183.8405 | 0.6799 | 0.4168 | 0.5326 | 0.6025 | 0.5233 | 15.9963 |
| 16.689 | 4.9598 | 1750 | 187.9473 | -88.4162 | -1.2945 | 183.8510 | 0.6799 | 0.4168 | 0.5326 | 0.6025 | 0.5233 | 15.9953 |
### Framework versions
- Transformers 4.42.0
- Pytorch 2.3.0+cu121
- Datasets 3.2.0
- Tokenizers 0.19.1
|