Model save

README.md

@@ -1,11 +1,10 @@
 ---
 base_model: meta-llama/Meta-Llama-3.1-8B
 datasets:
-- llama-duo/synth_classification_dataset_dedup
+- generator
 library_name: peft
 license: llama3.1
 tags:
-- alignment-handbook
 - trl
 - sft
 - generated_from_trainer
@@ -19,9 +18,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # llama3.1-8b-classification-gpt4o-100k
 
-This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B) on the llama-duo/synth_classification_dataset_dedup dataset.
+This model is a fine-tuned version of [meta-llama/Meta-Llama-3.1-8B](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.8520
+- Loss: 3.0330
 
 ## Model description
 
@@ -56,18 +55,18 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 1.4961 | 0.9978 | 225 | 1.7708 |
-| 1.3952 | 2.0 | 451 | 1.7770 |
-| 1.3491 | 2.9978 | 676 | 1.7484 |
-| 1.3025 | 4.0 | 902 | 1.7902 |
-| 1.2904 | 4.9978 | 1127 | 1.7997 |
-| 1.2729 | 6.0 | 1353 | 1.8170 |
-| 1.2451 | 6.9978 | 1578 | 1.8180 |
-| 1.229 | 8.0 | 1804 | 1.8372 |
-| 1.2239 | 8.9978 | 2029 | 1.8482 |
-| 1.2051 | 9.9778 | 2250 | 1.8520 |
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 1.2062 | 1.0 | 296 | 1.6781 |
+| 1.1339 | 2.0 | 592 | 1.6897 |
+| 1.0779 | 3.0 | 888 | 1.7536 |
+| 1.0043 | 4.0 | 1184 | 1.8225 |
+| 0.9288 | 5.0 | 1480 | 2.0044 |
+| 0.8437 | 6.0 | 1776 | 2.1710 |
+| 0.7654 | 7.0 | 2072 | 2.4080 |
+| 0.7117 | 8.0 | 2368 | 2.6554 |
+| 0.6916 | 9.0 | 2664 | 2.9172 |
+| 0.6652 | 10.0 | 2960 | 3.0330 |
 
 
 ### Framework versions

all_results.json

@@ -1,14 +1,9 @@
 {
- "epoch": 9.977827050997783,
- "eval_loss": 1.8520119190216064,
- "eval_runtime": 0.3553,
- "eval_samples": 16,
- "eval_samples_per_second": 2.814,
- "eval_steps_per_second": 2.814,
- "total_flos": 3.3259687719144e+18,
- "train_loss": 1.3362829395929972,
- "train_runtime": 6815.0283,
+ "epoch": 10.0,
+ "total_flos": 4.416382035459834e+18,
+ "train_loss": 0.922980490487975,
+ "train_runtime": 12382.7598,
  "train_samples": 92634,
- "train_samples_per_second": 10.572,
- "train_steps_per_second": 0.33
+ "train_samples_per_second": 7.645,
+ "train_steps_per_second": 0.239
 }

train_results.json

@@ -1,9 +1,9 @@
 {
- "epoch": 9.977827050997783,
- "total_flos": 3.3259687719144e+18,
- "train_loss": 1.3362829395929972,
- "train_runtime": 6815.0283,
+ "epoch": 10.0,
+ "total_flos": 4.416382035459834e+18,
+ "train_loss": 0.922980490487975,
+ "train_runtime": 12382.7598,
  "train_samples": 92634,
- "train_samples_per_second": 10.572,
- "train_steps_per_second": 0.33
+ "train_samples_per_second": 7.645,
+ "train_steps_per_second": 0.239
 }