File size: 4,736 Bytes
ca1e571 c171adf ca1e571 c171adf ca1e571 c171adf ca1e571 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 |
---
license: apache-2.0
base_model: hZzy/qwen2.5-0.5b-sft-news-IFT
tags:
- alignment-handbook
- ndcg
- trl
- expo
- generated_from_trainer
- trl
- expo
- generated_from_trainer
datasets:
- hZzy/train_pairwise_weighted
model-index:
- name: qwen2.5-0.5b-expo-DPO-W2-noES6-1
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/zhiyuzha-university-of-florida/huggingface/runs/c6z04yeu)
# qwen2.5-0.5b-expo-DPO-W2-noES6-1
This model is a fine-tuned version of [hZzy/qwen2.5-0.5b-sft-news-IFT](https://huggingface.co/hZzy/qwen2.5-0.5b-sft-news-IFT) on the hZzy/train_pairwise_weighted dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1677
- Logps: -77.5515
- Logits: -1.0043
- Objective: 0.1588
- Regularize: 1.7957
- Ranking Simple: 0.5461
- Wo Beta: 6.9275
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 3
- gradient_accumulation_steps: 12
- total_train_batch_size: 144
- total_eval_batch_size: 12
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 3
### Training results
| Training Loss | Epoch | Step | Validation Loss | Logps | Logits | Objective | Regularize | Ranking Simple | Wo Beta |
|:-------------:|:------:|:----:|:---------------:|:--------:|:-------:|:---------:|:----------:|:--------------:|:-------:|
| 0.0717 | 0.1417 | 50 | 0.0827 | -89.4542 | -1.3980 | 0.0833 | 0.9434 | 0.5233 | 7.6909 |
| 0.0782 | 0.2834 | 100 | 0.1198 | -90.3233 | -1.3848 | 0.1299 | 1.2948 | 0.5274 | 7.5300 |
| 0.1041 | 0.4251 | 150 | 0.1430 | -80.8115 | -1.4142 | 0.1445 | 1.5652 | 0.5331 | 7.2053 |
| 0.103 | 0.5668 | 200 | 0.1575 | -79.3788 | -1.2549 | 0.1620 | 1.6985 | 0.5383 | 7.0079 |
| 0.1301 | 0.7085 | 250 | 0.1765 | -80.7283 | -1.2947 | 0.1779 | 1.9721 | 0.5373 | 7.3699 |
| 0.0929 | 0.8503 | 300 | 0.1742 | -83.1364 | -1.0915 | 0.1719 | 1.9650 | 0.5399 | 7.3246 |
| 0.092 | 0.9920 | 350 | 0.1930 | -78.9639 | -1.2151 | 0.1810 | 1.9965 | 0.5492 | 6.8384 |
| 0.0713 | 1.1337 | 400 | 0.1963 | -76.5565 | -1.1860 | 0.1929 | 2.1515 | 0.5399 | 7.1718 |
| 0.0243 | 1.2754 | 450 | 0.1856 | -78.4444 | -1.1245 | 0.1782 | 2.0181 | 0.5414 | 7.0177 |
| 0.0514 | 1.4171 | 500 | 0.1857 | -77.6606 | -1.1929 | 0.1755 | 1.9383 | 0.5393 | 6.9356 |
| 0.0577 | 1.5588 | 550 | 0.1760 | -79.1478 | -1.0419 | 0.1699 | 1.8917 | 0.5450 | 6.9556 |
| 0.0391 | 1.7005 | 600 | 0.1791 | -80.1474 | -0.8913 | 0.1668 | 1.9362 | 0.5461 | 6.8670 |
| 0.0392 | 1.8422 | 650 | 0.1726 | -78.0514 | -0.9358 | 0.1615 | 1.8093 | 0.5512 | 6.8786 |
| 0.0385 | 1.9839 | 700 | 0.1687 | -77.0163 | -1.0116 | 0.1563 | 1.8321 | 0.5471 | 6.9309 |
| 0.0198 | 2.1256 | 750 | 0.1707 | -78.2445 | -1.0465 | 0.1584 | 1.8388 | 0.5492 | 6.8687 |
| 0.0072 | 2.2674 | 800 | 0.1708 | -78.1994 | -1.0332 | 0.1614 | 1.8241 | 0.5461 | 6.8566 |
| 0.0128 | 2.4091 | 850 | 0.1695 | -77.6488 | -0.9753 | 0.1603 | 1.8026 | 0.5487 | 6.8586 |
| 0.0105 | 2.5508 | 900 | 0.1680 | -77.8885 | -1.0018 | 0.1587 | 1.8027 | 0.5461 | 6.9417 |
| 0.0111 | 2.6925 | 950 | 0.1676 | -77.6180 | -1.0011 | 0.1585 | 1.8000 | 0.5466 | 6.9417 |
| 0.0122 | 2.8342 | 1000 | 0.1676 | -77.5617 | -1.0044 | 0.1588 | 1.7963 | 0.5461 | 6.9304 |
| 0.0117 | 2.9759 | 1050 | 0.1677 | -77.5515 | -1.0043 | 0.1588 | 1.7957 | 0.5461 | 6.9275 |
### Framework versions
- Transformers 4.42.0
- Pytorch 2.3.0+cu121
- Datasets 3.2.0
- Tokenizers 0.19.1
|