lightblue
/

qwen2.5-7B-instruct-kto

@@ -1,83 +1,97 @@
----
-library_name: transformers
-license: other
-base_model: Qwen/Qwen2.5-7B-Instruct
-tags:
-- llama-factory
-- full
-- generated_from_trainer
-model-index:
-- name: kto_trained_1
-  results: []
----
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-# kto_trained_1
-This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the lightblue_kto_data dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.3031
-- Rewards/chosen: 1.5421
-- Logps/chosen: -343.9051
-- Logits/chosen: -69679219.2
-- Rewards/rejected: -7.3046
-- Logps/rejected: -233.7684
-- Logits/rejected: -34451756.1379
-- Rewards/margins: 8.8467
-- Kl: 1080.3173
-## Model description
-More information needed
-## Intended uses & limitations
-More information needed
-## Training and evaluation data
-More information needed
-## Training procedure
-### Training hyperparameters
-The following hyperparameters were used during training:
-- learning_rate: 5e-06
-- train_batch_size: 1
-- eval_batch_size: 1
-- seed: 42
-- distributed_type: multi-GPU
-- num_devices: 8
-- gradient_accumulation_steps: 16
-- total_train_batch_size: 128
-- total_eval_batch_size: 8
-- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: cosine
-- lr_scheduler_warmup_ratio: 0.01
-- num_epochs: 1.0
-### Training results
-| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Logps/chosen | Logits/chosen  | Rewards/rejected | Logps/rejected | Logits/rejected | Rewards/margins |           |
-|:-------------:|:------:|:----:|:---------------:|:--------------:|:------------:|:--------------:|:----------------:|:--------------:|:---------------:|:---------------:|:---------:|
-| 0.2623        | 0.0997 | 36   | 0.3340          | 1.3847         | -345.4796    | -55713169.0667 | -3.6384          | -197.1070      | -40055004.6897  | 5.0231          | 890.2159  |
-| 0.3222        | 0.1995 | 72   | 0.3273          | 1.5219         | -344.1068    | -61469499.7333 | -4.9277          | -209.9999      | -32503238.6207  | 6.4496          | 1189.5447 |
-| 0.3798        | 0.2992 | 108  | 0.3185          | 1.5573         | -343.7531    | -63003302.4    | -5.7081          | -217.8038      | -31597484.1379  | 7.2654          | 955.4995  |
-| 0.3755        | 0.3990 | 144  | 0.3016          | 0.8908         | -350.4181    | -63924428.8    | -6.8986          | -229.7092      | -27711788.1379  | 7.7895          | 705.8951  |
-| 0.3454        | 0.4987 | 180  | 0.3053          | 1.4481         | -344.8449    | -67193476.2667 | -6.5311          | -226.0336      | -37107747.3103  | 7.9792          | 836.6326  |
-| 0.2633        | 0.5984 | 216  | 0.3085          | 1.5864         | -343.4627    | -68801646.9333 | -6.4654          | -225.3766      | -37986458.4828  | 8.0517          | 974.3778  |
-| 0.2519        | 0.6982 | 252  | 0.3109          | 1.5635         | -343.6908    | -69407142.4    | -6.4303          | -225.0262      | -34758311.7241  | 7.9939          | 1106.7635 |
-| 0.2959        | 0.7979 | 288  | 0.3033          | 1.6631         | -342.6956    | -69444923.7333 | -7.0061          | -230.7837      | -36029797.5172  | 8.6691          | 1082.5067 |
-| 0.2921        | 0.8977 | 324  | 0.3022          | 1.4322         | -345.0042    | -69711099.7333 | -7.5841          | -236.5635      | -35742644.9655  | 9.0163          | 1047.6223 |
-| 0.3122        | 0.9974 | 360  | 0.3031          | 1.5421         | -343.9051    | -69679219.2    | -7.3046          | -233.7684      | -34451756.1379  | 8.8467          | 1080.3173 |
-### Framework versions
-- Transformers 4.46.1
-- Pytorch 2.4.0+cu121
-- Datasets 3.1.0
-- Tokenizers 0.20.3

+---
+library_name: transformers
+license: other
+base_model: Qwen/Qwen2.5-7B-Instruct
+tags:
+- llama-factory
+- full
+- generated_from_trainer
+language:
+- zho
+- eng
+- fra
+- spa
+- por
+- deu
+- ita
+- rus
+- jpn
+- kor
+- vie
+- tha
+- ara
+model-index:
+- name: kto_trained_1
+  results: []
+---
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+# kto_trained_1
+This model is a fine-tuned version of [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) on the lightblue_kto_data dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3031
+- Rewards/chosen: 1.5421
+- Logps/chosen: -343.9051
+- Logits/chosen: -69679219.2
+- Rewards/rejected: -7.3046
+- Logps/rejected: -233.7684
+- Logits/rejected: -34451756.1379
+- Rewards/margins: 8.8467
+- Kl: 1080.3173
+## Model description
+More information needed
+## Intended uses & limitations
+More information needed
+## Training and evaluation data
+More information needed
+## Training procedure
+### Training hyperparameters
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 1
+- eval_batch_size: 1
+- seed: 42
+- distributed_type: multi-GPU
+- num_devices: 8
+- gradient_accumulation_steps: 16
+- total_train_batch_size: 128
+- total_eval_batch_size: 8
+- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.01
+- num_epochs: 1.0
+### Training results
+| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Logps/chosen | Logits/chosen  | Rewards/rejected | Logps/rejected | Logits/rejected | Rewards/margins |           |
+|:-------------:|:------:|:----:|:---------------:|:--------------:|:------------:|:--------------:|:----------------:|:--------------:|:---------------:|:---------------:|:---------:|
+| 0.2623        | 0.0997 | 36   | 0.3340          | 1.3847         | -345.4796    | -55713169.0667 | -3.6384          | -197.1070      | -40055004.6897  | 5.0231          | 890.2159  |
+| 0.3222        | 0.1995 | 72   | 0.3273          | 1.5219         | -344.1068    | -61469499.7333 | -4.9277          | -209.9999      | -32503238.6207  | 6.4496          | 1189.5447 |
+| 0.3798        | 0.2992 | 108  | 0.3185          | 1.5573         | -343.7531    | -63003302.4    | -5.7081          | -217.8038      | -31597484.1379  | 7.2654          | 955.4995  |
+| 0.3755        | 0.3990 | 144  | 0.3016          | 0.8908         | -350.4181    | -63924428.8    | -6.8986          | -229.7092      | -27711788.1379  | 7.7895          | 705.8951  |
+| 0.3454        | 0.4987 | 180  | 0.3053          | 1.4481         | -344.8449    | -67193476.2667 | -6.5311          | -226.0336      | -37107747.3103  | 7.9792          | 836.6326  |
+| 0.2633        | 0.5984 | 216  | 0.3085          | 1.5864         | -343.4627    | -68801646.9333 | -6.4654          | -225.3766      | -37986458.4828  | 8.0517          | 974.3778  |
+| 0.2519        | 0.6982 | 252  | 0.3109          | 1.5635         | -343.6908    | -69407142.4    | -6.4303          | -225.0262      | -34758311.7241  | 7.9939          | 1106.7635 |
+| 0.2959        | 0.7979 | 288  | 0.3033          | 1.6631         | -342.6956    | -69444923.7333 | -7.0061          | -230.7837      | -36029797.5172  | 8.6691          | 1082.5067 |
+| 0.2921        | 0.8977 | 324  | 0.3022          | 1.4322         | -345.0042    | -69711099.7333 | -7.5841          | -236.5635      | -35742644.9655  | 9.0163          | 1047.6223 |
+| 0.3122        | 0.9974 | 360  | 0.3031          | 1.5421         | -343.9051    | -69679219.2    | -7.3046          | -233.7684      | -34451756.1379  | 8.8467          | 1080.3173 |
+### Framework versions
+- Transformers 4.46.1
+- Pytorch 2.4.0+cu121
+- Datasets 3.1.0
+- Tokenizers 0.20.3