Hugging Face's logo

lightblue
/
qwen2.5-7B-instruct-kto

Text Generation
Transformers
Safetensors
qwen2
llama-factory
full
Generated from Trainer
conversational
text-generation-inference
Model card Files Community
1
Lightblue-Dev lbourdois commited on
Commit
ba573f4
·
verified ·
1 Parent(s): 7a3ab06

Improve language tag (#1)

Browse files

- Improve language tag (eb0418921fcf0f5b6e2bf96188f6dd95dae1f4a2)


Co-authored-by: Loïck BOURDOIS <[email protected]>

Files changed (1) hide show
  1. README.md +97 -83
README.md CHANGED
@@ -1,83 +1,97 @@
1
- ---
2
- library_name: transformers
3
- license: other
4
- base_model: Qwen/Qwen2.5-7B-Instruct
5
- tags:
6
- - llama-factory
7
- - full
8
- - generated_from_trainer
9
- model-index:
10
- - name: kto_trained_1
11
- results: []
12
- ---
13
-
14
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
- should probably proofread and complete it, then remove this comment. -->
16
-
17
- # kto_trained_1
18
-
19
- This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the lightblue_kto_data dataset.
20
- It achieves the following results on the evaluation set:
21
- - Loss: 0.3031
22
- - Rewards/chosen: 1.5421
23
- - Logps/chosen: -343.9051
24
- - Logits/chosen: -69679219.2
25
- - Rewards/rejected: -7.3046
26
- - Logps/rejected: -233.7684
27
- - Logits/rejected: -34451756.1379
28
- - Rewards/margins: 8.8467
29
- - Kl: 1080.3173
30
-
31
- ## Model description
32
-
33
- More information needed
34
-
35
- ## Intended uses & limitations
36
-
37
- More information needed
38
-
39
- ## Training and evaluation data
40
-
41
- More information needed
42
-
43
- ## Training procedure
44
-
45
- ### Training hyperparameters
46
-
47
- The following hyperparameters were used during training:
48
- - learning_rate: 5e-06
49
- - train_batch_size: 1
50
- - eval_batch_size: 1
51
- - seed: 42
52
- - distributed_type: multi-GPU
53
- - num_devices: 8
54
- - gradient_accumulation_steps: 16
55
- - total_train_batch_size: 128
56
- - total_eval_batch_size: 8
57
- - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
58
- - lr_scheduler_type: cosine
59
- - lr_scheduler_warmup_ratio: 0.01
60
- - num_epochs: 1.0
61
-
62
- ### Training results
63
-
64
- | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Logps/chosen | Logits/chosen | Rewards/rejected | Logps/rejected | Logits/rejected | Rewards/margins | |
65
- |:-------------:|:------:|:----:|:---------------:|:--------------:|:------------:|:--------------:|:----------------:|:--------------:|:---------------:|:---------------:|:---------:|
66
- | 0.2623 | 0.0997 | 36 | 0.3340 | 1.3847 | -345.4796 | -55713169.0667 | -3.6384 | -197.1070 | -40055004.6897 | 5.0231 | 890.2159 |
67
- | 0.3222 | 0.1995 | 72 | 0.3273 | 1.5219 | -344.1068 | -61469499.7333 | -4.9277 | -209.9999 | -32503238.6207 | 6.4496 | 1189.5447 |
68
- | 0.3798 | 0.2992 | 108 | 0.3185 | 1.5573 | -343.7531 | -63003302.4 | -5.7081 | -217.8038 | -31597484.1379 | 7.2654 | 955.4995 |
69
- | 0.3755 | 0.3990 | 144 | 0.3016 | 0.8908 | -350.4181 | -63924428.8 | -6.8986 | -229.7092 | -27711788.1379 | 7.7895 | 705.8951 |
70
- | 0.3454 | 0.4987 | 180 | 0.3053 | 1.4481 | -344.8449 | -67193476.2667 | -6.5311 | -226.0336 | -37107747.3103 | 7.9792 | 836.6326 |
71
- | 0.2633 | 0.5984 | 216 | 0.3085 | 1.5864 | -343.4627 | -68801646.9333 | -6.4654 | -225.3766 | -37986458.4828 | 8.0517 | 974.3778 |
72
- | 0.2519 | 0.6982 | 252 | 0.3109 | 1.5635 | -343.6908 | -69407142.4 | -6.4303 | -225.0262 | -34758311.7241 | 7.9939 | 1106.7635 |
73
- | 0.2959 | 0.7979 | 288 | 0.3033 | 1.6631 | -342.6956 | -69444923.7333 | -7.0061 | -230.7837 | -36029797.5172 | 8.6691 | 1082.5067 |
74
- | 0.2921 | 0.8977 | 324 | 0.3022 | 1.4322 | -345.0042 | -69711099.7333 | -7.5841 | -236.5635 | -35742644.9655 | 9.0163 | 1047.6223 |
75
- | 0.3122 | 0.9974 | 360 | 0.3031 | 1.5421 | -343.9051 | -69679219.2 | -7.3046 | -233.7684 | -34451756.1379 | 8.8467 | 1080.3173 |
76
-
77
-
78
- ### Framework versions
79
-
80
- - Transformers 4.46.1
81
- - Pytorch 2.4.0+cu121
82
- - Datasets 3.1.0
83
- - Tokenizers 0.20.3
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: other
4
+ base_model: Qwen/Qwen2.5-7B-Instruct
5
+ tags:
6
+ - llama-factory
7
+ - full
8
+ - generated_from_trainer
9
+ language:
10
+ - zho
11
+ - eng
12
+ - fra
13
+ - spa
14
+ - por
15
+ - deu
16
+ - ita
17
+ - rus
18
+ - jpn
19
+ - kor
20
+ - vie
21
+ - tha
22
+ - ara
23
+ model-index:
24
+ - name: kto_trained_1
25
+ results: []
26
+ ---
27
+
28
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
29
+ should probably proofread and complete it, then remove this comment. -->
30
+
31
+ # kto_trained_1
32
+
33
+ This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the lightblue_kto_data dataset.
34
+ It achieves the following results on the evaluation set:
35
+ - Loss: 0.3031
36
+ - Rewards/chosen: 1.5421
37
+ - Logps/chosen: -343.9051
38
+ - Logits/chosen: -69679219.2
39
+ - Rewards/rejected: -7.3046
40
+ - Logps/rejected: -233.7684
41
+ - Logits/rejected: -34451756.1379
42
+ - Rewards/margins: 8.8467
43
+ - Kl: 1080.3173
44
+
45
+ ## Model description
46
+
47
+ More information needed
48
+
49
+ ## Intended uses & limitations
50
+
51
+ More information needed
52
+
53
+ ## Training and evaluation data
54
+
55
+ More information needed
56
+
57
+ ## Training procedure
58
+
59
+ ### Training hyperparameters
60
+
61
+ The following hyperparameters were used during training:
62
+ - learning_rate: 5e-06
63
+ - train_batch_size: 1
64
+ - eval_batch_size: 1
65
+ - seed: 42
66
+ - distributed_type: multi-GPU
67
+ - num_devices: 8
68
+ - gradient_accumulation_steps: 16
69
+ - total_train_batch_size: 128
70
+ - total_eval_batch_size: 8
71
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
72
+ - lr_scheduler_type: cosine
73
+ - lr_scheduler_warmup_ratio: 0.01
74
+ - num_epochs: 1.0
75
+
76
+ ### Training results
77
+
78
+ | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Logps/chosen | Logits/chosen | Rewards/rejected | Logps/rejected | Logits/rejected | Rewards/margins | |
79
+ |:-------------:|:------:|:----:|:---------------:|:--------------:|:------------:|:--------------:|:----------------:|:--------------:|:---------------:|:---------------:|:---------:|
80
+ | 0.2623 | 0.0997 | 36 | 0.3340 | 1.3847 | -345.4796 | -55713169.0667 | -3.6384 | -197.1070 | -40055004.6897 | 5.0231 | 890.2159 |
81
+ | 0.3222 | 0.1995 | 72 | 0.3273 | 1.5219 | -344.1068 | -61469499.7333 | -4.9277 | -209.9999 | -32503238.6207 | 6.4496 | 1189.5447 |
82
+ | 0.3798 | 0.2992 | 108 | 0.3185 | 1.5573 | -343.7531 | -63003302.4 | -5.7081 | -217.8038 | -31597484.1379 | 7.2654 | 955.4995 |
83
+ | 0.3755 | 0.3990 | 144 | 0.3016 | 0.8908 | -350.4181 | -63924428.8 | -6.8986 | -229.7092 | -27711788.1379 | 7.7895 | 705.8951 |
84
+ | 0.3454 | 0.4987 | 180 | 0.3053 | 1.4481 | -344.8449 | -67193476.2667 | -6.5311 | -226.0336 | -37107747.3103 | 7.9792 | 836.6326 |
85
+ | 0.2633 | 0.5984 | 216 | 0.3085 | 1.5864 | -343.4627 | -68801646.9333 | -6.4654 | -225.3766 | -37986458.4828 | 8.0517 | 974.3778 |
86
+ | 0.2519 | 0.6982 | 252 | 0.3109 | 1.5635 | -343.6908 | -69407142.4 | -6.4303 | -225.0262 | -34758311.7241 | 7.9939 | 1106.7635 |
87
+ | 0.2959 | 0.7979 | 288 | 0.3033 | 1.6631 | -342.6956 | -69444923.7333 | -7.0061 | -230.7837 | -36029797.5172 | 8.6691 | 1082.5067 |
88
+ | 0.2921 | 0.8977 | 324 | 0.3022 | 1.4322 | -345.0042 | -69711099.7333 | -7.5841 | -236.5635 | -35742644.9655 | 9.0163 | 1047.6223 |
89
+ | 0.3122 | 0.9974 | 360 | 0.3031 | 1.5421 | -343.9051 | -69679219.2 | -7.3046 | -233.7684 | -34451756.1379 | 8.8467 | 1080.3173 |
90
+
91
+
92
+ ### Framework versions
93
+
94
+ - Transformers 4.46.1
95
+ - Pytorch 2.4.0+cu121
96
+ - Datasets 3.1.0
97
+ - Tokenizers 0.20.3