pszemraj commited on
Commit
c6cfc9d
·
verified ·
1 Parent(s): 3b6f2da

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -18
README.md CHANGED
@@ -28,17 +28,26 @@ It achieves the following results on the evaluation set:
28
  - Gen Len: 441.475
29
  - Num Input Tokens Seen: 435513684
30
 
31
- ## Model description
32
-
33
- More information needed
34
-
35
- ## Intended uses & limitations
36
-
37
- More information needed
38
-
39
- ## Training and evaluation data
40
-
41
- More information needed
 
 
 
 
 
 
 
 
 
42
 
43
  ## Training procedure
44
 
@@ -73,10 +82,3 @@ The following hyperparameters were used during training:
73
  | 1.2577 | 0.8874 | 11000 | 1.1752 | 39.3539 | 23.0123 | 31.9005 | 37.4941 | 424.445 | 386471860 |
74
  | 1.193 | 0.9680 | 12000 | 1.1526 | 40.1804 | 23.1008 | 32.3484 | 38.2103 | 422.225 | 421585440 |
75
 
76
-
77
- ### Framework versions
78
-
79
- - Transformers 4.45.1
80
- - Pytorch 2.4.1+cu124
81
- - Datasets 3.0.1
82
- - Tokenizers 0.20.0
 
28
  - Gen Len: 441.475
29
  - Num Input Tokens Seen: 435513684
30
 
31
+ ## Quick eval
32
+
33
+ Quick eval for: `pszemraj/tFINE-850m-24x24-v0.5-instruct-L1`
34
+
35
+
36
+ hf (pretrained=pszemraj/tFINE-850m-24x24-v0.5-instruct-L1,trust_remote_code=True,dtype=bfloat16,trust_remote_code=True), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8
37
+ | Tasks |Version| Filter |n-shot| Metric | |Value | |Stderr|
38
+ |-------------|------:|----------------|-----:|-----------|---|-----:|---|------|
39
+ |boolq | 2|none | 0|acc |↑ |0.5661|± |0.0087|
40
+ |openbookqa | 1|none | 0|acc |↑ |0.1540|± |0.0162|
41
+ | | |none | 0|acc_norm |↑ |0.2960|± |0.0204|
42
+ |piqa | 1|none | 0|acc |↑ |0.6094|± |0.0114|
43
+ | | |none | 0|acc_norm |↑ |0.5952|± |0.0115|
44
+ |social_iqa | 0|none | 0|acc |↑ |0.3900|± |0.0110|
45
+ |tinyArc | 0|none | 25|acc_norm |↑ |0.2903|± | N/A|
46
+ |tinyGSM8k | 0|flexible-extract| 5|exact_match|↑ |0.0471|± | N/A|
47
+ | | |strict-match | 5|exact_match|↑ |0.0339|± | N/A|
48
+ |tinyHellaswag| 0|none | 10|acc_norm |↑ |0.2490|± | N/A|
49
+ |tinyMMLU | 0|none | 0|acc_norm |↑ |0.3021|± | N/A|
50
+ |winogrande | 1|none | 0|acc |↑ |0.4925|± |0.0141|
51
 
52
  ## Training procedure
53
 
 
82
  | 1.2577 | 0.8874 | 11000 | 1.1752 | 39.3539 | 23.0123 | 31.9005 | 37.4941 | 424.445 | 386471860 |
83
  | 1.193 | 0.9680 | 12000 | 1.1526 | 40.1804 | 23.1008 | 32.3484 | 38.2103 | 422.225 | 421585440 |
84