Hugging Face's logo

hZzy
/
qwen2.5-0.5b-expo-L2EXPO-ES-0.01

Safetensors
qwen2
alignment-handbook
ndcg
trl
expo
Generated from Trainer
Model card Files Community
hZzy commited on
Commit
e0d3a9c
•
1 Parent(s): d373b1a

Model save

Browse files
Files changed (6) hide show
  1. README.md +80 -0
  2. all_results.json +9 -0
  3. generation_config.json +7 -0
  4. model.safetensors +1 -1
  5. train_results.json +9 -0
  6. trainer_state.json +265 -0
README.md ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: hZzy/qwen2.5-0.5b-sft-news-IFT
4
+ tags:
5
+ - trl
6
+ - expo
7
+ - generated_from_trainer
8
+ model-index:
9
+ - name: qwen2.5-0.5b-expo-L2EXPO-ES-0.01
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ [<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/zhiyuzha-university-of-florida/huggingface/runs/8q1vl3fv)
17
+ # qwen2.5-0.5b-expo-L2EXPO-ES-0.01
18
+
19
+ This model is a fine-tuned version of [hZzy/qwen2.5-0.5b-sft-news-IFT](https://huggingface.co/hZzy/qwen2.5-0.5b-sft-news-IFT) on an unknown dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.3975
22
+ - Logps: -123.3866
23
+ - Logits: -2.2621
24
+ - Objective: 0.3953
25
+ - Dpo Loss: 0.6769
26
+ - Regularize: 0.3953
27
+ - Ranking Simple: 0.5740
28
+ - Ranking Idealized: 0.8732
29
+ - Ranking Idealized Expo: 0.5321
30
+ - Wo Beta: 23.6726
31
+
32
+ ## Model description
33
+
34
+ More information needed
35
+
36
+ ## Intended uses & limitations
37
+
38
+ More information needed
39
+
40
+ ## Training and evaluation data
41
+
42
+ More information needed
43
+
44
+ ## Training procedure
45
+
46
+ ### Training hyperparameters
47
+
48
+ The following hyperparameters were used during training:
49
+ - learning_rate: 5e-06
50
+ - train_batch_size: 4
51
+ - eval_batch_size: 4
52
+ - seed: 42
53
+ - distributed_type: multi-GPU
54
+ - num_devices: 3
55
+ - gradient_accumulation_steps: 12
56
+ - total_train_batch_size: 144
57
+ - total_eval_batch_size: 12
58
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
59
+ - lr_scheduler_type: cosine
60
+ - lr_scheduler_warmup_ratio: 0.1
61
+ - num_epochs: 5
62
+
63
+ ### Training results
64
+
65
+ | Training Loss | Epoch | Step | Validation Loss | Logps | Logits | Objective | Dpo Loss | Regularize | Ranking Simple | Ranking Idealized | Ranking Idealized Expo | Wo Beta |
66
+ |:-------------:|:------:|:----:|:---------------:|:---------:|:-------:|:---------:|:--------:|:----------:|:--------------:|:-----------------:|:----------------------:|:-------:|
67
+ | 0.4144 | 0.1417 | 50 | 0.4116 | -92.3444 | -1.6667 | 0.4097 | 0.6906 | 0.4097 | 0.5290 | 0.8732 | 0.5321 | 17.6306 |
68
+ | 0.394 | 0.2834 | 100 | 0.4078 | -115.8757 | -2.0312 | 0.4074 | 0.6863 | 0.4074 | 0.5399 | 0.8732 | 0.5321 | 22.1058 |
69
+ | 0.3504 | 0.4251 | 150 | 0.4044 | -123.1670 | -2.0505 | 0.4018 | 0.6803 | 0.4018 | 0.5719 | 0.8732 | 0.5321 | 23.5660 |
70
+ | 0.3135 | 0.5668 | 200 | 0.4006 | -121.6031 | -2.1409 | 0.3974 | 0.6781 | 0.3974 | 0.5621 | 0.8732 | 0.5321 | 23.0977 |
71
+ | 0.2807 | 0.7085 | 250 | 0.4043 | -122.1639 | -2.3711 | 0.4010 | 0.6790 | 0.4010 | 0.5600 | 0.8732 | 0.5321 | 23.6688 |
72
+ | 0.2532 | 0.8503 | 300 | 0.3975 | -123.3866 | -2.2621 | 0.3953 | 0.6769 | 0.3953 | 0.5740 | 0.8732 | 0.5321 | 23.6726 |
73
+
74
+
75
+ ### Framework versions
76
+
77
+ - Transformers 4.42.0
78
+ - Pytorch 2.3.0+cu121
79
+ - Datasets 2.19.1
80
+ - Tokenizers 0.19.1
all_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.8502598016060463,
3
+ "total_flos": 0.0,
4
+ "train_loss": 0.3342974317073822,
5
+ "train_runtime": 8266.6282,
6
+ "train_samples": 50802,
7
+ "train_samples_per_second": 30.727,
8
+ "train_steps_per_second": 0.213
9
+ }
generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 151644,
3
+ "eos_token_id": 151645,
4
+ "max_new_tokens": 2048,
5
+ "pad_token_id": 151645,
6
+ "transformers_version": "4.42.0"
7
+ }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2884b3a8e848332a4ba7298fb21bda12f6a96bc00a0783faa3f39245102c113e
3
  size 1975192208
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9a3a71ac6667901c61a909de38365be83331607e1ccb226566998dc0006083b6
3
  size 1975192208
train_results.json ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "epoch": 0.8502598016060463,
3
+ "total_flos": 0.0,
4
+ "train_loss": 0.3342974317073822,
5
+ "train_runtime": 8266.6282,
6
+ "train_samples": 50802,
7
+ "train_samples_per_second": 30.727,
8
+ "train_steps_per_second": 0.213
9
+ }
trainer_state.json ADDED
@@ -0,0 +1,265 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "best_metric": 17.63062286376953,
3
+ "best_model_checkpoint": "./qwen2.5-0.5b/qwen2.5-0.5b-expo-L2EXPO-ES-0.01/checkpoint-50",
4
+ "epoch": 0.8502598016060463,
5
+ "eval_steps": 50,
6
+ "global_step": 300,
7
+ "is_hyper_param_search": false,
8
+ "is_local_process_zero": true,
9
+ "is_world_process_zero": true,
10
+ "log_history": [
11
+ {
12
+ "dpo_loss": 0.6931471824645996,
13
+ "epoch": 0.002834199338686821,
14
+ "grad_norm": 3.6884903788633325,
15
+ "learning_rate": 2.840909090909091e-08,
16
+ "logits": -1.359458565711975,
17
+ "logps": -84.69721221923828,
18
+ "loss": 0.3913,
19
+ "objective": 0.3618059456348419,
20
+ "ranking_idealized": 0.875,
21
+ "ranking_idealized_expo": 0.5833333134651184,
22
+ "ranking_simple": 0.5833333134651184,
23
+ "regularize": 0.3618059456348419,
24
+ "step": 1,
25
+ "wo_beta": 14.830931663513184
26
+ },
27
+ {
28
+ "dpo_loss": 0.6914557218551636,
29
+ "epoch": 0.14170996693434104,
30
+ "grad_norm": 4.3876993528860435,
31
+ "learning_rate": 1.4204545454545458e-06,
32
+ "logits": -1.524561882019043,
33
+ "logps": -82.59858703613281,
34
+ "loss": 0.4144,
35
+ "objective": 0.41888776421546936,
36
+ "ranking_idealized": 0.8945578336715698,
37
+ "ranking_idealized_expo": 0.5348639488220215,
38
+ "ranking_simple": 0.5284863710403442,
39
+ "regularize": 0.4188877046108246,
40
+ "step": 50,
41
+ "wo_beta": 16.066608428955078
42
+ },
43
+ {
44
+ "epoch": 0.14170996693434104,
45
+ "eval_dpo_loss": 0.6905742287635803,
46
+ "eval_logits": -1.6666735410690308,
47
+ "eval_logps": -92.34442901611328,
48
+ "eval_loss": 0.41158440709114075,
49
+ "eval_objective": 0.4097130298614502,
50
+ "eval_ranking_idealized": 0.8731883764266968,
51
+ "eval_ranking_idealized_expo": 0.5320910811424255,
52
+ "eval_ranking_simple": 0.5289855003356934,
53
+ "eval_regularize": 0.4097130298614502,
54
+ "eval_runtime": 308.7557,
55
+ "eval_samples_per_second": 18.753,
56
+ "eval_steps_per_second": 1.564,
57
+ "eval_wo_beta": 17.63062286376953,
58
+ "step": 50
59
+ },
60
+ {
61
+ "dpo_loss": 0.6717649102210999,
62
+ "epoch": 0.2834199338686821,
63
+ "grad_norm": 9.179969250495992,
64
+ "learning_rate": 2.8409090909090916e-06,
65
+ "logits": -1.801299810409546,
66
+ "logps": -98.4464111328125,
67
+ "loss": 0.394,
68
+ "objective": 0.39795053005218506,
69
+ "ranking_idealized": 0.8958333134651184,
70
+ "ranking_idealized_expo": 0.5270833373069763,
71
+ "ranking_simple": 0.5887500047683716,
72
+ "regularize": 0.39795053005218506,
73
+ "step": 100,
74
+ "wo_beta": 20.196491241455078
75
+ },
76
+ {
77
+ "epoch": 0.2834199338686821,
78
+ "eval_dpo_loss": 0.6862898468971252,
79
+ "eval_logits": -2.0312013626098633,
80
+ "eval_logps": -115.87567138671875,
81
+ "eval_loss": 0.4077964425086975,
82
+ "eval_objective": 0.40739303827285767,
83
+ "eval_ranking_idealized": 0.8731883764266968,
84
+ "eval_ranking_idealized_expo": 0.5320910811424255,
85
+ "eval_ranking_simple": 0.5398550629615784,
86
+ "eval_regularize": 0.40739303827285767,
87
+ "eval_runtime": 307.8646,
88
+ "eval_samples_per_second": 18.807,
89
+ "eval_steps_per_second": 1.569,
90
+ "eval_wo_beta": 22.105756759643555,
91
+ "step": 100
92
+ },
93
+ {
94
+ "dpo_loss": 0.6482623219490051,
95
+ "epoch": 0.42512990080302315,
96
+ "grad_norm": 9.270844197401257,
97
+ "learning_rate": 4.2613636363636365e-06,
98
+ "logits": -1.8636983633041382,
99
+ "logps": -110.8066177368164,
100
+ "loss": 0.3504,
101
+ "objective": 0.35595157742500305,
102
+ "ranking_idealized": 0.8862500190734863,
103
+ "ranking_idealized_expo": 0.5358333587646484,
104
+ "ranking_simple": 0.6287500262260437,
105
+ "regularize": 0.35595157742500305,
106
+ "step": 150,
107
+ "wo_beta": 25.672834396362305
108
+ },
109
+ {
110
+ "epoch": 0.42512990080302315,
111
+ "eval_dpo_loss": 0.6803079843521118,
112
+ "eval_logits": -2.0504531860351562,
113
+ "eval_logps": -123.1670150756836,
114
+ "eval_loss": 0.4044354259967804,
115
+ "eval_objective": 0.4018007516860962,
116
+ "eval_ranking_idealized": 0.8731883764266968,
117
+ "eval_ranking_idealized_expo": 0.5320910811424255,
118
+ "eval_ranking_simple": 0.5719461441040039,
119
+ "eval_regularize": 0.4018007516860962,
120
+ "eval_runtime": 308.5087,
121
+ "eval_samples_per_second": 18.768,
122
+ "eval_steps_per_second": 1.566,
123
+ "eval_wo_beta": 23.56603240966797,
124
+ "step": 150
125
+ },
126
+ {
127
+ "dpo_loss": 0.6220158934593201,
128
+ "epoch": 0.5668398677373642,
129
+ "grad_norm": 8.001451658093092,
130
+ "learning_rate": 4.997168347957521e-06,
131
+ "logits": -1.9987245798110962,
132
+ "logps": -114.0056381225586,
133
+ "loss": 0.3135,
134
+ "objective": 0.3158811330795288,
135
+ "ranking_idealized": 0.8883333206176758,
136
+ "ranking_idealized_expo": 0.527916669845581,
137
+ "ranking_simple": 0.67208331823349,
138
+ "regularize": 0.3158811330795288,
139
+ "step": 200,
140
+ "wo_beta": 29.25323486328125
141
+ },
142
+ {
143
+ "epoch": 0.5668398677373642,
144
+ "eval_dpo_loss": 0.678061842918396,
145
+ "eval_logits": -2.1408538818359375,
146
+ "eval_logps": -121.60306549072266,
147
+ "eval_loss": 0.4005555212497711,
148
+ "eval_objective": 0.397381991147995,
149
+ "eval_ranking_idealized": 0.8731883764266968,
150
+ "eval_ranking_idealized_expo": 0.5320910811424255,
151
+ "eval_ranking_simple": 0.5621117949485779,
152
+ "eval_regularize": 0.397381991147995,
153
+ "eval_runtime": 307.9864,
154
+ "eval_samples_per_second": 18.8,
155
+ "eval_steps_per_second": 1.568,
156
+ "eval_wo_beta": 23.097728729248047,
157
+ "step": 200
158
+ },
159
+ {
160
+ "dpo_loss": 0.5992652773857117,
161
+ "epoch": 0.7085498346717053,
162
+ "grad_norm": 8.50379539811797,
163
+ "learning_rate": 4.973122855144066e-06,
164
+ "logits": -2.132956027984619,
165
+ "logps": -119.59600067138672,
166
+ "loss": 0.2807,
167
+ "objective": 0.28073957562446594,
168
+ "ranking_idealized": 0.8787500262260437,
169
+ "ranking_idealized_expo": 0.5266666412353516,
170
+ "ranking_simple": 0.71875,
171
+ "regularize": 0.28073957562446594,
172
+ "step": 250,
173
+ "wo_beta": 33.336734771728516
174
+ },
175
+ {
176
+ "epoch": 0.7085498346717053,
177
+ "eval_dpo_loss": 0.6789642572402954,
178
+ "eval_logits": -2.3711183071136475,
179
+ "eval_logps": -122.16390228271484,
180
+ "eval_loss": 0.4042646586894989,
181
+ "eval_objective": 0.4010438024997711,
182
+ "eval_ranking_idealized": 0.8731883764266968,
183
+ "eval_ranking_idealized_expo": 0.5320910811424255,
184
+ "eval_ranking_simple": 0.5600414276123047,
185
+ "eval_regularize": 0.4010438024997711,
186
+ "eval_runtime": 308.0112,
187
+ "eval_samples_per_second": 18.798,
188
+ "eval_steps_per_second": 1.568,
189
+ "eval_wo_beta": 23.668811798095703,
190
+ "step": 250
191
+ },
192
+ {
193
+ "dpo_loss": 0.5932748317718506,
194
+ "epoch": 0.8502598016060463,
195
+ "grad_norm": 7.481377422405124,
196
+ "learning_rate": 4.924776641419513e-06,
197
+ "logits": -2.1575162410736084,
198
+ "logps": -116.334716796875,
199
+ "loss": 0.2532,
200
+ "objective": 0.2530641555786133,
201
+ "ranking_idealized": 0.8733333349227905,
202
+ "ranking_idealized_expo": 0.5104166865348816,
203
+ "ranking_simple": 0.7279166579246521,
204
+ "regularize": 0.2530641555786133,
205
+ "step": 300,
206
+ "wo_beta": 33.5853271484375
207
+ },
208
+ {
209
+ "epoch": 0.8502598016060463,
210
+ "eval_dpo_loss": 0.6769444346427917,
211
+ "eval_logits": -2.2620654106140137,
212
+ "eval_logps": -123.38658905029297,
213
+ "eval_loss": 0.3974739611148834,
214
+ "eval_objective": 0.3952558934688568,
215
+ "eval_ranking_idealized": 0.8731883764266968,
216
+ "eval_ranking_idealized_expo": 0.5320910811424255,
217
+ "eval_ranking_simple": 0.5740165710449219,
218
+ "eval_regularize": 0.3952558934688568,
219
+ "eval_runtime": 308.2767,
220
+ "eval_samples_per_second": 18.782,
221
+ "eval_steps_per_second": 1.567,
222
+ "eval_wo_beta": 23.67264175415039,
223
+ "step": 300
224
+ },
225
+ {
226
+ "epoch": 0.8502598016060463,
227
+ "step": 300,
228
+ "total_flos": 0.0,
229
+ "train_loss": 0.3342974317073822,
230
+ "train_runtime": 8266.6282,
231
+ "train_samples_per_second": 30.727,
232
+ "train_steps_per_second": 0.213
233
+ }
234
+ ],
235
+ "logging_steps": 50,
236
+ "max_steps": 1760,
237
+ "num_input_tokens_seen": 0,
238
+ "num_train_epochs": 5,
239
+ "save_steps": 50,
240
+ "stateful_callbacks": {
241
+ "EarlyStoppingCallback": {
242
+ "args": {
243
+ "early_stopping_patience": 5,
244
+ "early_stopping_threshold": 0.0
245
+ },
246
+ "attributes": {
247
+ "early_stopping_patience_counter": 0
248
+ }
249
+ },
250
+ "TrainerControl": {
251
+ "args": {
252
+ "should_epoch_stop": false,
253
+ "should_evaluate": false,
254
+ "should_log": false,
255
+ "should_save": true,
256
+ "should_training_stop": true
257
+ },
258
+ "attributes": {}
259
+ }
260
+ },
261
+ "total_flos": 0.0,
262
+ "train_batch_size": 4,
263
+ "trial_name": null,
264
+ "trial_params": null
265
+ }