Crystalcareai commited on
Commit
44add4f
1 Parent(s): e44bfe5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -9
README.md CHANGED
@@ -19,15 +19,16 @@ The model was trained using a state-of-the-art distillation pipeline and an inst
19
  Llama-3.1-SuperNova-Lite excels in both benchmark performance and real-world applications, providing the power of large-scale models in a more compact, efficient form ideal for organizations seeking high performance with reduced resource requirements.
20
 
21
  # Evaluations
22
- We will be submitting this model to the OpenLLM Leaderboard for a more conclusive benchmark - but here are our internal benchmarks (these will be updated as they come in):
23
-
24
- | Benchmark | Score |
25
- |-----------|-------|
26
- | IF_Eval | 81.1 |
27
- | MMLU Pro | 38.7 |
28
- | TruthfulQA| 64.4 |
29
- | BBH | 51.1 |
30
- | GPQA | 31.2 |
 
31
 
32
 
33
  # note
 
19
  Llama-3.1-SuperNova-Lite excels in both benchmark performance and real-world applications, providing the power of large-scale models in a more compact, efficient form ideal for organizations seeking high performance with reduced resource requirements.
20
 
21
  # Evaluations
22
+ We will be submitting this model to the OpenLLM Leaderboard for a more conclusive benchmark - but here are our internal benchmarks using the main branch of lm evaluation harness:
23
+
24
+ | Benchmark | SuperNova-Lite | Llama-3.1-8b-Instruct |
25
+ |-------------|----------------|----------------------|
26
+ | IF_Eval | 81.1 | 77.4 |
27
+ | MMLU Pro | 38.7 | 37.7 |
28
+ | TruthfulQA | 64.4 | 55.0 |
29
+ | BBH | 51.1 | 50.6 |
30
+ | GPQA | 31.2 | 29.02 |
31
+
32
 
33
 
34
  # note