Update README.md
Browse files
README.md
CHANGED
@@ -28,17 +28,26 @@ It achieves the following results on the evaluation set:
|
|
28 |
- Gen Len: 441.475
|
29 |
- Num Input Tokens Seen: 435513684
|
30 |
|
31 |
-
##
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
42 |
|
43 |
## Training procedure
|
44 |
|
@@ -73,10 +82,3 @@ The following hyperparameters were used during training:
|
|
73 |
| 1.2577 | 0.8874 | 11000 | 1.1752 | 39.3539 | 23.0123 | 31.9005 | 37.4941 | 424.445 | 386471860 |
|
74 |
| 1.193 | 0.9680 | 12000 | 1.1526 | 40.1804 | 23.1008 | 32.3484 | 38.2103 | 422.225 | 421585440 |
|
75 |
|
76 |
-
|
77 |
-
### Framework versions
|
78 |
-
|
79 |
-
- Transformers 4.45.1
|
80 |
-
- Pytorch 2.4.1+cu124
|
81 |
-
- Datasets 3.0.1
|
82 |
-
- Tokenizers 0.20.0
|
|
|
28 |
- Gen Len: 441.475
|
29 |
- Num Input Tokens Seen: 435513684
|
30 |
|
31 |
+
## Quick eval
|
32 |
+
|
33 |
+
Quick eval for: `pszemraj/tFINE-850m-24x24-v0.5-instruct-L1`
|
34 |
+
|
35 |
+
|
36 |
+
hf (pretrained=pszemraj/tFINE-850m-24x24-v0.5-instruct-L1,trust_remote_code=True,dtype=bfloat16,trust_remote_code=True), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8
|
37 |
+
| Tasks |Version| Filter |n-shot| Metric | |Value | |Stderr|
|
38 |
+
|-------------|------:|----------------|-----:|-----------|---|-----:|---|------|
|
39 |
+
|boolq | 2|none | 0|acc |↑ |0.5661|± |0.0087|
|
40 |
+
|openbookqa | 1|none | 0|acc |↑ |0.1540|± |0.0162|
|
41 |
+
| | |none | 0|acc_norm |↑ |0.2960|± |0.0204|
|
42 |
+
|piqa | 1|none | 0|acc |↑ |0.6094|± |0.0114|
|
43 |
+
| | |none | 0|acc_norm |↑ |0.5952|± |0.0115|
|
44 |
+
|social_iqa | 0|none | 0|acc |↑ |0.3900|± |0.0110|
|
45 |
+
|tinyArc | 0|none | 25|acc_norm |↑ |0.2903|± | N/A|
|
46 |
+
|tinyGSM8k | 0|flexible-extract| 5|exact_match|↑ |0.0471|± | N/A|
|
47 |
+
| | |strict-match | 5|exact_match|↑ |0.0339|± | N/A|
|
48 |
+
|tinyHellaswag| 0|none | 10|acc_norm |↑ |0.2490|± | N/A|
|
49 |
+
|tinyMMLU | 0|none | 0|acc_norm |↑ |0.3021|± | N/A|
|
50 |
+
|winogrande | 1|none | 0|acc |↑ |0.4925|± |0.0141|
|
51 |
|
52 |
## Training procedure
|
53 |
|
|
|
82 |
| 1.2577 | 0.8874 | 11000 | 1.1752 | 39.3539 | 23.0123 | 31.9005 | 37.4941 | 424.445 | 386471860 |
|
83 |
| 1.193 | 0.9680 | 12000 | 1.1526 | 40.1804 | 23.1008 | 32.3484 | 38.2103 | 422.225 | 421585440 |
|
84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|