Crystalcareai
commited on
Commit
•
44add4f
1
Parent(s):
e44bfe5
Update README.md
Browse files
README.md
CHANGED
@@ -19,15 +19,16 @@ The model was trained using a state-of-the-art distillation pipeline and an inst
|
|
19 |
Llama-3.1-SuperNova-Lite excels in both benchmark performance and real-world applications, providing the power of large-scale models in a more compact, efficient form ideal for organizations seeking high performance with reduced resource requirements.
|
20 |
|
21 |
# Evaluations
|
22 |
-
We will be submitting this model to the OpenLLM Leaderboard for a more conclusive benchmark - but here are our internal benchmarks
|
23 |
-
|
24 |
-
| Benchmark |
|
25 |
-
|
26 |
-
| IF_Eval
|
27 |
-
| MMLU Pro
|
28 |
-
| TruthfulQA| 64.4
|
29 |
-
| BBH
|
30 |
-
| GPQA
|
|
|
31 |
|
32 |
|
33 |
# note
|
|
|
19 |
Llama-3.1-SuperNova-Lite excels in both benchmark performance and real-world applications, providing the power of large-scale models in a more compact, efficient form ideal for organizations seeking high performance with reduced resource requirements.
|
20 |
|
21 |
# Evaluations
|
22 |
+
We will be submitting this model to the OpenLLM Leaderboard for a more conclusive benchmark - but here are our internal benchmarks using the main branch of lm evaluation harness:
|
23 |
+
|
24 |
+
| Benchmark | SuperNova-Lite | Llama-3.1-8b-Instruct |
|
25 |
+
|-------------|----------------|----------------------|
|
26 |
+
| IF_Eval | 81.1 | 77.4 |
|
27 |
+
| MMLU Pro | 38.7 | 37.7 |
|
28 |
+
| TruthfulQA | 64.4 | 55.0 |
|
29 |
+
| BBH | 51.1 | 50.6 |
|
30 |
+
| GPQA | 31.2 | 29.02 |
|
31 |
+
|
32 |
|
33 |
|
34 |
# note
|