Hugging Face's logo

hugosousa
/
SmolLM2-1.7B-TemporalQuestions

Text Classification
Transformers
Safetensors
llama
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Community
hugosousa commited on
Commit
4051c53
·
verified ·
1 Parent(s): 7e2f34a

Model save

Browse files
Files changed (2) hide show
  1. README.md +96 -0
  2. model.safetensors +1 -1
README.md ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: HuggingFaceTB/SmolLM2-1.7B
5
+ tags:
6
+ - generated_from_trainer
7
+ metrics:
8
+ - f1
9
+ model-index:
10
+ - name: SmolLM2-1.7B-TemporalQuestions
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # SmolLM2-1.7B-TemporalQuestions
18
+
19
+ This model is a fine-tuned version of [HuggingFaceTB/SmolLM2-1.7B](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B) on an unknown dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 0.0398
22
+ - F1: 0.9874
23
+
24
+ ## Model description
25
+
26
+ More information needed
27
+
28
+ ## Intended uses & limitations
29
+
30
+ More information needed
31
+
32
+ ## Training and evaluation data
33
+
34
+ More information needed
35
+
36
+ ## Training procedure
37
+
38
+ ### Training hyperparameters
39
+
40
+ The following hyperparameters were used during training:
41
+ - learning_rate: 0.001
42
+ - train_batch_size: 4
43
+ - eval_batch_size: 4
44
+ - seed: 42
45
+ - distributed_type: multi-GPU
46
+ - num_devices: 4
47
+ - gradient_accumulation_steps: 128
48
+ - total_train_batch_size: 2048
49
+ - total_eval_batch_size: 16
50
+ - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
51
+ - lr_scheduler_type: cosine_with_restarts
52
+ - lr_scheduler_warmup_ratio: 0.05
53
+ - num_epochs: 30
54
+
55
+ ### Training results
56
+
57
+ | Training Loss | Epoch | Step | Validation Loss | F1 |
58
+ |:-------------:|:-------:|:----:|:---------------:|:------:|
59
+ | 0.2142 | 1.0 | 112 | 0.1175 | 0.9066 |
60
+ | 0.1218 | 2.0 | 224 | 0.0498 | 0.9606 |
61
+ | 0.0219 | 3.0 | 336 | 0.0363 | 0.9755 |
62
+ | 0.1146 | 4.0 | 448 | 0.0350 | 0.9749 |
63
+ | 0.0062 | 5.0 | 560 | 0.0297 | 0.9775 |
64
+ | 0.079 | 6.0 | 672 | 0.0361 | 0.9767 |
65
+ | 0.0261 | 7.0 | 784 | 0.0297 | 0.9818 |
66
+ | 0.0225 | 8.0 | 896 | 0.0290 | 0.9804 |
67
+ | 0.0004 | 9.0 | 1008 | 0.0308 | 0.9816 |
68
+ | 0.0105 | 10.0 | 1120 | 0.0333 | 0.9829 |
69
+ | 0.0014 | 11.0 | 1232 | 0.0287 | 0.9837 |
70
+ | 0.0035 | 12.0 | 1344 | 0.0301 | 0.9848 |
71
+ | 0.0035 | 13.0 | 1456 | 0.0290 | 0.9835 |
72
+ | 0.0785 | 14.0 | 1568 | 0.0291 | 0.9846 |
73
+ | 0.0014 | 15.0 | 1680 | 0.0303 | 0.9840 |
74
+ | 0.0003 | 16.0 | 1792 | 0.0356 | 0.9837 |
75
+ | 0.0002 | 17.0 | 1904 | 0.0347 | 0.9862 |
76
+ | 0.0003 | 18.0 | 2016 | 0.0338 | 0.9865 |
77
+ | 0.0002 | 19.0 | 2128 | 0.0335 | 0.9872 |
78
+ | 0.0001 | 20.0 | 2240 | 0.0352 | 0.9878 |
79
+ | 0.0001 | 21.0 | 2352 | 0.0362 | 0.9878 |
80
+ | 0.0001 | 22.0 | 2464 | 0.0378 | 0.9875 |
81
+ | 0.0001 | 23.0 | 2576 | 0.0386 | 0.9874 |
82
+ | 0.0001 | 24.0 | 2688 | 0.0391 | 0.9873 |
83
+ | 0.0001 | 25.0 | 2800 | 0.0394 | 0.9874 |
84
+ | 0.0001 | 26.0 | 2912 | 0.0396 | 0.9874 |
85
+ | 0.0001 | 27.0 | 3024 | 0.0397 | 0.9874 |
86
+ | 0.0001 | 28.0 | 3136 | 0.0397 | 0.9874 |
87
+ | 0.0001 | 29.0 | 3248 | 0.0398 | 0.9874 |
88
+ | 0.0036 | 29.7385 | 3330 | 0.0398 | 0.9874 |
89
+
90
+
91
+ ### Framework versions
92
+
93
+ - Transformers 4.47.1
94
+ - Pytorch 2.5.1+cu124
95
+ - Datasets 3.0.1
96
+ - Tokenizers 0.21.0
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:99f72583702a39babef9f4773ee9f2c27d0322f99362fd8ef4cf8b7653b99506
3
  size 3422827192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f43ba1d8be57609e89e2307fec591b31cd559d002485ca7346656ffd1686bf71
3
  size 3422827192