MHGanainy/mgpt-lora-multi-shared-1024

README.md

@@ -15,6 +15,8 @@ should probably proofread and complete it, then remove this comment. -->
 # mgpt-lora-multi-shared-1024
 
 This model is a fine-tuned version of [ai-forever/mGPT](https://huggingface.co/ai-forever/mGPT) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.7504
 
 ## Model description
 

all_results.json

@@ -1,13 +1,13 @@
 {
     "epoch": 1.0,
-    "eval_loss": 3.800776720046997,
-    "eval_runtime": 10.5511,
-    "eval_samples_per_second": 9.478,
-    "eval_steps_per_second": 9.478,
-    "perplexity": 44.73591828689196,
-    "total_flos": 1527651080601600.0,
-    "train_loss": 4.172185363769532,
-    "train_runtime": 144.7805,
-    "train_samples_per_second": 0.691,
-    "train_steps_per_second": 0.691
+    "eval_loss": 1.750352382659912,
+    "eval_runtime": 1628.7902,
+    "eval_samples_per_second": 76.532,
+    "eval_steps_per_second": 9.567,
+    "perplexity": 5.7566308555301,
+    "total_flos": 7.631778497299481e+18,
+    "train_loss": 2.1485319636500972,
+    "train_runtime": 14119.5859,
+    "train_samples_per_second": 35.382,
+    "train_steps_per_second": 4.423
 }

eval_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 1.0,
-    "eval_loss": 3.800776720046997,
-    "eval_runtime": 10.5511,
-    "eval_samples_per_second": 9.478,
-    "eval_steps_per_second": 9.478,
-    "perplexity": 44.73591828689196
+    "eval_loss": 1.750352382659912,
+    "eval_runtime": 1628.7902,
+    "eval_samples_per_second": 76.532,
+    "eval_steps_per_second": 9.567,
+    "perplexity": 5.7566308555301
 }

train_results.json

@@ -1,8 +1,8 @@
 {
     "epoch": 1.0,
-    "total_flos": 1527651080601600.0,
-    "train_loss": 4.172185363769532,
-    "train_runtime": 144.7805,
-    "train_samples_per_second": 0.691,
-    "train_steps_per_second": 0.691
+    "total_flos": 7.631778497299481e+18,
+    "train_loss": 2.1485319636500972,
+    "train_runtime": 14119.5859,
+    "train_samples_per_second": 35.382,
+    "train_steps_per_second": 4.423
 }