Update README.md
Browse files
README.md
CHANGED
|
@@ -24,6 +24,41 @@ Original model: https://huggingface.co/google/gemma-2-9b-it
|
|
| 24 |
|
| 25 |
All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
|
| 26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
<hr><br>
|
| 28 |
|
| 29 |
# Gemma 2 model card
|
|
|
|
| 24 |
|
| 25 |
All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
|
| 26 |
|
| 27 |
+
<hr>
|
| 28 |
+
|
| 29 |
+
# Perplexity table (the lower the better)
|
| 30 |
+
|
| 31 |
+
| Quant | Size (MB) | Perplexity (PPL) | Size (%) | Accuracy (%) | PPL Error rate |
|
| 32 |
+
| ------- | --------- | ---------------- | -------- | ------------ | -------------- |
|
| 33 |
+
| IQ1_S | 2269 | 16.0064 | 12.87 | \-45.09 | 0.12077 |
|
| 34 |
+
| IQ1_M | 2429 | 13.7255 | 13.77 | \-35.97 | 0.10272 |
|
| 35 |
+
| IQ2_XXS | 2695 | 11.269 | 15.28 | \-22.01 | 0.08345 |
|
| 36 |
+
| IQ2_XS | 2926 | 10.5628 | 16.59 | \-16.8 | 0.07809 |
|
| 37 |
+
| IQ2_S | 3063 | 10.3671 | 17.37 | \-15.23 | 0.07772 |
|
| 38 |
+
| IQ2_M | 3276 | 9.7973 | 18.58 | \-10.3 | 0.07298 |
|
| 39 |
+
| Q2_K_S | 3388 | 9.9206 | 19.21 | \-11.41 | 0.07247 |
|
| 40 |
+
| IQ3_XXS | 3621 | 9.3955 | 20.53 | \-6.46 | 0.06962 |
|
| 41 |
+
| Q2_K | 3630 | 9.421 | 20.58 | \-6.71 | 0.0683 |
|
| 42 |
+
| IQ3_XS | 3953 | 9.2545 | 22.42 | \-5.04 | 0.06868 |
|
| 43 |
+
| IQ3_S | 4137 | 9.2127 | 23.46 | \-4.61 | 0.06866 |
|
| 44 |
+
| Q3_K_S | 4137 | 9.083 | 23.46 | \-3.24 | 0.06618 |
|
| 45 |
+
| IQ3_M | 4287 | 8.9791 | 24.31 | \-2.12 | 0.06614 |
|
| 46 |
+
| Q3_K_M | 4542 | 9.0172 | 25.76 | \-2.54 | 0.06684 |
|
| 47 |
+
| Q3_K_L | 4895 | 8.9965 | 27.76 | \-2.31 | 0.06675 |
|
| 48 |
+
| IQ4_XS | 4943 | 8.8286 | 28.03 | \-0.46 | 0.06504 |
|
| 49 |
+
| IQ4_NL | 5191 | 8.8235 | 29.44 | \-0.4 | 0.06496 |
|
| 50 |
+
| Q4_0 | 5207 | 8.834 | 29.53 | \-0.52 | 0.0648 |
|
| 51 |
+
| Q4_K_S | 5226 | 8.829 | 29.63 | \-0.46 | 0.06513 |
|
| 52 |
+
| Q4_K_M | 5495 | 8.8069 | 31.16 | \-0.21 | 0.06493 |
|
| 53 |
+
| Q4_1 | 5688 | 8.8395 | 32.25 | \-0.58 | 0.06526 |
|
| 54 |
+
| Q5_K_S | 6184 | 8.8011 | 35.07 | \-0.14 | 0.06504 |
|
| 55 |
+
| Q5_0 | 6199 | 8.7668 | 35.15 | 0.25 | 0.06455 |
|
| 56 |
+
| Q5_K_M | 6340 | 8.7993 | 35.95 | \-0.12 | 0.06506 |
|
| 57 |
+
| Q5_1 | 6680 | 8.7888 | 37.88 | 0 | 0.06493 |
|
| 58 |
+
| Q6_K | 7238 | 8.7863 | 41.04 | 0.02 | 0.06497 |
|
| 59 |
+
| Q8_0 | 9372 | 8.7858 | 53.14 | 0.03 | 0.06497 |
|
| 60 |
+
| F16 | 17635 | 8.7884 | 100 | 0 | 0.06501 |
|
| 61 |
+
|
| 62 |
<hr><br>
|
| 63 |
|
| 64 |
# Gemma 2 model card
|