Model save

README.md

@@ -1,13 +1,13 @@
 ---
 base_model: facebook/opt-350m
 datasets:
-- HuggingFaceH4/ultrachat_200k
+- generator
 library_name: peft
 license: other
 tags:
-- alignment-handbook
 - trl
 - sft
+- alignment-handbook
 - generated_from_trainer
 model-index:
 - name: opt350
@@ -19,9 +19,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 # opt350
 
-This model is a fine-tuned version of [facebook/opt-350m](https://huggingface.co/facebook/opt-350m) on the HuggingFaceH4/ultrachat_200k dataset.
-It achieves the following results on the evaluation set:
-- Loss: 1.7869
+This model is a fine-tuned version of [facebook/opt-350m](https://huggingface.co/facebook/opt-350m) on the generator dataset.
 
 ## Model description
 
@@ -56,9 +54,6 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 1.8289        | 0.9999 | 8068 | 1.7869          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,13 +20,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "v_proj",
     "down_proj",
-    "gate_proj",
     "k_proj",
-    "up_proj",
+    "gate_proj",
     "o_proj",
-    "q_proj",
-    "v_proj"
+    "up_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

all_results.json

@@ -6,9 +6,9 @@
     "eval_samples_per_second": 98.221,
     "eval_steps_per_second": 6.143,
     "total_flos": 4248917998829568.0,
-    "train_loss": 1.8640377591255577,
-    "train_runtime": 7899.8856,
+    "train_loss": 0.0,
+    "train_runtime": 0.013,
     "train_samples": 207864,
-    "train_samples_per_second": 16.341,
-    "train_steps_per_second": 1.021
+    "train_samples_per_second": 9898425.558,
+    "train_steps_per_second": 618618.052
 }

runs/Nov06_12-35-29_gnode001.cluster/events.out.tfevents.1730925336.gnode001.cluster.319001.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83ace51b3dce42ebf39b6f376bc63f4eec7936b16f30fbec200f44e9e0eb13b3
+size 5968

train_results.json

@@ -1,9 +1,9 @@
 {
     "epoch": 0.9999380306128772,
     "total_flos": 4248917998829568.0,
-    "train_loss": 1.8640377591255577,
-    "train_runtime": 7899.8856,
+    "train_loss": 0.0,
+    "train_runtime": 0.013,
     "train_samples": 207864,
-    "train_samples_per_second": 16.341,
-    "train_steps_per_second": 1.021
+    "train_samples_per_second": 9898425.558,
+    "train_steps_per_second": 618618.052
 }

trainer_state.json

@@ -11306,22 +11306,14 @@
       "loss": 1.8289,
       "step": 8065
     },
-    {
-      "epoch": 0.9999380306128772,
-      "eval_loss": 1.7869144678115845,
-      "eval_runtime": 146.7975,
-      "eval_samples_per_second": 97.27,
-      "eval_steps_per_second": 6.083,
-      "step": 8068
-    },
     {
       "epoch": 0.9999380306128772,
       "step": 8068,
       "total_flos": 4248917998829568.0,
-      "train_loss": 1.8640377591255577,
-      "train_runtime": 7899.8856,
-      "train_samples_per_second": 16.341,
-      "train_steps_per_second": 1.021
+      "train_loss": 0.0,
+      "train_runtime": 0.013,
+      "train_samples_per_second": 9898425.558,
+      "train_steps_per_second": 618618.052
     }
   ],
   "logging_steps": 5,

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bf597ff109e8eb432c6a2edc1b25470f5aa9cb37ee0e041326691c23bae90273
+oid sha256:00d22c401c4097001dc3bef189c4e94267b1e886a53abe7580d57bab88f54a8b
 size 7032