End of training

README.md

@@ -18,8 +18,8 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [unsloth/llama-3-8b-bnb-4bit](https://huggingface.co/unsloth/llama-3-8b-bnb-4bit) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.0751
-- Accuracy: 0.6294
+- Loss: 0.8012
+- Accuracy: 0.7706
 
 ## Model description
 
@@ -39,24 +39,26 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.0001
-- train_batch_size: 8
-- eval_batch_size: 8
+- train_batch_size: 4
+- eval_batch_size: 4
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 1
+- num_epochs: 2
+- mixed_precision_training: Native AMP
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | Accuracy |
 |:-------------:|:-----:|:----:|:---------------:|:--------:|
-| 0.8852        | 1.0   | 150  | 1.0751          | 0.6294   |
+| 0.6763        | 1.0   | 300  | 0.7377          | 0.7529   |
+| 0.419         | 2.0   | 600  | 0.8012          | 0.7706   |
 
 
 ### Framework versions
 
 - PEFT 0.10.0
-- Transformers 4.40.1
+- Transformers 4.40.2
 - Pytorch 2.2.1+cu121
 - Datasets 2.19.1
 - Tokenizers 0.19.1

adapter_config.json

@@ -20,8 +20,8 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "o_proj",
     "q_proj",
+    "o_proj",
     "v_proj",
     "k_proj"
   ],

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2e03c844df5243479fa21f5ba45329e874cd64102846b5e78a9bc19d0dad14c0
+oid sha256:962a9e234ffbbc2e97f1e4f02a3dc217b65aa2b557b915614fbb8ee896913c0d
 size 54593240

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:edb144c36d359049026d22dd341214f71c1f47689ea1cfa0e9d3cfb4115ea671
+oid sha256:a499dce6018a4128b8c579e7ebc8a45ba118dde6e56078381e3cee105c33620c
 size 5048