Hugging Face's logo

pmorelr
/
Mistral-7B-v0.1-ICD-Expert-QLoRA-test-2

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files Metrics Community
pmorelr commited on
Commit
9c769bf
•
1 Parent(s): bb2421d

Model save

Browse files
Files changed (6) hide show
  1. README.md +5 -5
  2. adapter_config.json +2 -2
  3. adapter_model.safetensors +2 -2
  4. runs/Jan26_08-38-47_pedro-morel-rosa-workspace-cuda-0/events.out.tfevents.1706258573.pedro-morel-rosa-workspace-cuda-0.1047.1 +3 -0
  5. runs/Jan26_10-08-57_pedro-morel-rosa-workspace-cuda-0/events.out.tfevents.1706263757.pedro-morel-rosa-workspace-cuda-0.1735.0 +3 -0
  6. training_args.bin +1 -1
README.md CHANGED
@@ -20,7 +20,7 @@ should probably proofread and complete it, then remove this comment. -->
20
 
21
  This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the generator dataset.
22
  It achieves the following results on the evaluation set:
23
- - Loss: 1.5534
24
 
25
  ## Model description
26
 
@@ -54,10 +54,10 @@ The following hyperparameters were used during training:
54
 
55
  | Training Loss | Epoch | Step | Validation Loss |
56
  |:-------------:|:-----:|:----:|:---------------:|
57
- | 1.8212 | 0.97 | 18 | 1.7546 |
58
- | 1.6305 | 1.99 | 37 | 1.6045 |
59
- | 1.5805 | 2.96 | 55 | 1.5591 |
60
- | 1.5579 | 3.88 | 72 | 1.5534 |
61
 
62
 
63
  ### Framework versions
 
20
 
21
  This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the generator dataset.
22
  It achieves the following results on the evaluation set:
23
+ - Loss: 1.6641
24
 
25
  ## Model description
26
 
 
54
 
55
  | Training Loss | Epoch | Step | Validation Loss |
56
  |:-------------:|:-----:|:----:|:---------------:|
57
+ | 1.8895 | 0.97 | 18 | 1.8383 |
58
+ | 1.749 | 1.99 | 37 | 1.7245 |
59
+ | 1.6948 | 2.96 | 55 | 1.6720 |
60
+ | 1.6684 | 3.88 | 72 | 1.6641 |
61
 
62
 
63
  ### Framework versions
adapter_config.json CHANGED
@@ -9,13 +9,13 @@
9
  "layers_pattern": null,
10
  "layers_to_transform": null,
11
  "loftq_config": {},
12
- "lora_alpha": 64,
13
  "lora_dropout": 0.1,
14
  "megatron_config": null,
15
  "megatron_core": "megatron.core",
16
  "modules_to_save": null,
17
  "peft_type": "LORA",
18
- "r": 128,
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
 
9
  "layers_pattern": null,
10
  "layers_to_transform": null,
11
  "loftq_config": {},
12
+ "lora_alpha": 32,
13
  "lora_dropout": 0.1,
14
  "megatron_config": null,
15
  "megatron_core": "megatron.core",
16
  "modules_to_save": null,
17
  "peft_type": "LORA",
18
+ "r": 64,
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0c341a178bc67585cb14e59d2ca720ca273ecadad3c9b8e74b088399475c074c
3
- size 436242776
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:74ccd31679e07065274ccad7c12a8f69a4b3fefa72a411964a7a4e8a455b1d0a
3
+ size 218138576
runs/Jan26_08-38-47_pedro-morel-rosa-workspace-cuda-0/events.out.tfevents.1706258573.pedro-morel-rosa-workspace-cuda-0.1047.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f2aae107d2c7ff75e416e29dc627091849e810de76e553182ea53c35d273e4f
3
+ size 5405
runs/Jan26_10-08-57_pedro-morel-rosa-workspace-cuda-0/events.out.tfevents.1706263757.pedro-morel-rosa-workspace-cuda-0.1735.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e50134e6384b8f1b7440b9518833c5d19a5c1635bf55dcbf0d59c3b17e07947a
3
+ size 9127
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d8fe4796ddb8c437ba51b7be4550a6de44c4579075525eb284de0b5ac7e60fa9
3
  size 4856
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c3895989c66e37b62a0af070fdab10058c70081ca593c278e61da71cd3e9e94
3
  size 4856