lillybak/mistral-7binstruct-summary-100s

README.md

@@ -20,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) on the generator dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.4320
+- Loss: 1.5927
 
 ## Model description
 
@@ -45,21 +45,20 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
-- lr_scheduler_warmup_steps: 0.03
-- training_steps: 50
+- lr_scheduler_warmup_steps: 2
+- training_steps: 10
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 1.7045        | 0.21  | 25   | 1.4648          |
-| 1.4977        | 0.42  | 50   | 1.4320          |
+| 1.7463        | 0.08  | 10   | 1.5927          |
 
 
 ### Framework versions
 
-- PEFT 0.8.2
-- Transformers 4.38.1
+- PEFT 0.9.0
+- Transformers 4.38.2
 - Pytorch 2.1.0+cu121
 - Datasets 2.17.1
 - Tokenizers 0.15.2

adapter_config.json

@@ -23,5 +23,6 @@
     "q_proj"
   ],
   "task_type": "CAUSAL_LM",
+  "use_dora": false,
   "use_rslora": false
 }

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:72f8d0829bfba6ad670139958e8acf51a6358827a1054144de274465442c5701
-size 27280152
+oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
+size 48

runs/Mar01_16-43-40_8bda7af05057/events.out.tfevents.1709311432.8bda7af05057.1220.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21d39f7fb6ecbd62fe418fee3b27360b487393bc33f8248675e358da71cd67a1
+size 6696

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4d93c9523e0c8d78ceed3e7ec17329e826eb405df17e30586ae5464d096d8bc9
+oid sha256:55c4f97650ed8a551a28581e94f3826e47349cfe3e95b6faf15633e7ececa099
 size 4920