Added evaluation metrics
Browse files
README.md
CHANGED
|
@@ -14,4 +14,7 @@ Official [AQLM](https://arxiv.org/abs/2401.06118) quantization of [meta-llama/Me
|
|
| 14 |
For this quantization, we used 1 codebook of 16 bits.
|
| 15 |
|
| 16 |
Results:
|
| 17 |
-
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
For this quantization, we used 1 codebook of 16 bits.
|
| 15 |
|
| 16 |
Results:
|
| 17 |
+
| Model | Quantization | MMLU (5-shot) | GSM8k (8-shot) | Model size, Gb |
|
| 18 |
+
|------|------|-------|------|------|
|
| 19 |
+
|meta-llama/Meta-Llama-3-8B-Instruct| None |0.6560 | 0.7475 | 16.1 |
|
| 20 |
+
| | 1x16 | 0.5529 | 0.3397 | 4.1 |
|