| ## Quick eval | |
| Quick eval for: `BEE-spoke-data/tFINE-900m-e16-d32-instruct_2e` | |
| hf (pretrained=BEE-spoke-data/tFINE-900m-e16-d32-instruct_2e,trust_remote_code=True,dtype=bfloat16,trust_remote_code=True), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8 | |
| | Tasks |Version| Filter |n-shot| Metric | |Value | |Stderr| | |
| |-------------|------:|----------------|-----:|-----------|---|-----:|---|------| | |
| |boolq | 2|none | 0|acc |↑ |0.6254|± |0.0085| | |
| |openbookqa | 1|none | 0|acc |↑ |0.1520|± |0.0161| | |
| | | |none | 0|acc_norm |↑ |0.3100|± |0.0207| | |
| |piqa | 1|none | 0|acc |↑ |0.6072|± |0.0114| | |
| | | |none | 0|acc_norm |↑ |0.5996|± |0.0114| | |
| |social_iqa | 0|none | 0|acc |↑ |0.4212|± |0.0112| | |
| |tinyArc | 0|none | 25|acc_norm |↑ |0.2998|± | N/A| | |
| |tinyGSM8k | 0|flexible-extract| 5|exact_match|↑ |0.0605|± | N/A| | |
| | | |strict-match | 5|exact_match|↑ |0.0432|± | N/A| | |
| |tinyHellaswag| 0|none | 10|acc_norm |↑ |0.2969|± | N/A| | |
| |tinyMMLU | 0|none | 0|acc_norm |↑ |0.3120|± | N/A| | |
| |winogrande | 1|none | 0|acc |↑ |0.4964|± |0.0141| |