File size: 4,476 Bytes
073e9a9 8ac4c6a 073e9a9 8ac4c6a 073e9a9 8ac4c6a 073e9a9 8ac4c6a 073e9a9 8ac4c6a 073e9a9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 |
---
license: apache-2.0
base_model: hZzy/qwen2.5-0.5b-sft-news-IFT
tags:
- alignment-handbook
- ndcg
- trl
- expo
- generated_from_trainer
- trl
- expo
- generated_from_trainer
datasets:
- hZzy/train_pairwise_weighted
model-index:
- name: qwen2.5-0.5b-expo-L2EXPO-ES-10
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/zhiyuzha-university-of-florida/huggingface/runs/akswogl4)
# qwen2.5-0.5b-expo-L2EXPO-ES-10
This model is a fine-tuned version of [hZzy/qwen2.5-0.5b-sft-news-IFT](https://huggingface.co/hZzy/qwen2.5-0.5b-sft-news-IFT) on the hZzy/train_pairwise_weighted dataset.
It achieves the following results on the evaluation set:
- Loss: 38.5263
- Logps: -75.2715
- Logits: -0.8236
- Objective: 37.4366
- Dpo Loss: 18.9116
- Regularize: 37.4366
- Ranking Simple: 0.5295
- Ranking Idealized: 0.5212
- Ranking Idealized Expo: 0.5212
- Wo Beta: 14.6816
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 3
- gradient_accumulation_steps: 12
- total_train_batch_size: 144
- total_eval_batch_size: 12
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 5
### Training results
| Training Loss | Epoch | Step | Dpo Loss | Logits | Logps | Validation Loss | Objective | Ranking Idealized | Ranking Idealized Expo | Ranking Simple | Regularize | Wo Beta |
|:-------------:|:------:|:----:|:--------:|:-------:|:--------:|:---------------:|:---------:|:-----------------:|:----------------------:|:--------------:|:----------:|:-------:|
| 4.4544 | 0.1417 | 50 | 3.0769 | -1.4000 | -90.0695 | 6.1132 | 6.2354 | 0.5212 | 0.5212 | 0.5243 | 6.2354 | 16.0705 |
| 17.3779 | 0.2834 | 100 | 7.9374 | -1.3238 | -85.5257 | 16.1760 | 16.0037 | 0.5212 | 0.5212 | 0.5259 | 16.0037 | 15.7780 |
| 28.1478 | 0.4251 | 150 | 14.7239 | -1.0824 | -82.4808 | 28.7309 | 28.1308 | 0.5212 | 0.5212 | 0.5228 | 28.1308 | 15.4096 |
| 35.2522 | 0.5668 | 200 | 18.9116 | -0.8236 | -75.2715 | 38.5263 | 37.4366 | 0.5212 | 0.5212 | 0.5295 | 37.4366 | 14.6816 |
| 37.8556 | 0.7085 | 250 | 22.7495 | -0.6024 | -76.2798 | 44.8164 | 44.5795 | 0.5212 | 0.5212 | 0.5223 | 44.5795 | 14.3182 |
| 36.0351 | 0.8503 | 300 | 22.1457 | -0.7057 | -79.1833 | 44.3831 | 43.8777 | 0.5212 | 0.5212 | 0.5254 | 43.8777 | 14.2675 |
| 32.9882 | 0.9920 | 350 | 23.0098 | -0.6345 | -80.3166 | 46.6946 | 45.5953 | 0.5212 | 0.5212 | 0.5248 | 45.5953 | 14.1690 |
| 30.7247 | 1.1337 | 400 | 48.3805 | -82.4111| -0.4810 | 48.0656 | 24.6183 | 48.0656 | 0.5166 | 0.5212 | 0.5212 | 14.1059 |
| 29.6491 | 1.2754 | 450 | 48.5237 | -81.5285| -0.5861 | 48.8411 | 24.9495 | 48.8411 | 0.5243 | 0.5212 | 0.5212 | 14.4793 |
| 28.3933 | 1.4171 | 500 | 47.8150 | -79.8843| -0.5585 | 47.9210 | 24.8156 | 47.9210 | 0.5212 | 0.5212 | 0.5212 | 14.3458 |
| 26.3026 | 1.5588 | 550 | 48.0081 | -79.5567| -0.5594 | 48.2215 | 24.4583 | 48.2215 | 0.5228 | 0.5212 | 0.5212 | 14.1587 |
| 25.1162 | 1.7005 | 600 | 49.4271 | -79.4245| -0.4875 | 49.7428 | 25.2219 | 49.7428 | 0.5259 | 0.5212 | 0.5212 | 14.1923 |
### Framework versions
- Transformers 4.42.0
- Pytorch 2.3.0+cu121
- Datasets 3.2.0
- Tokenizers 0.19.1
|