Hugging Face's logo

ThomasBaruzier
/
gemma-2-9b-it-GGUF

Text Generation
Transformers
GGUF
conversational
Inference Endpoints
Model card Files Community
ThomasBaruzier commited on
Commit
18c7c8e
·
verified ·
1 Parent(s): 0097d26

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +28 -28
README.md CHANGED
@@ -30,34 +30,34 @@ All quants were made using the imatrix option and Bartowski's [calibration file]
30
 
31
  | Quant | Size (MB) | Perplexity (PPL) | Size (%) | Accuracy (%) | PPL Error rate |
32
  | ------- | --------- | ---------------- | -------- | ------------ | -------------- |
33
- | IQ1_S | 2269 | 16.0064 | 12.87 | \-45.09 | 0.12077 |
34
- | IQ1_M | 2429 | 13.7255 | 13.77 | \-35.97 | 0.10272 |
35
- | IQ2_XXS | 2695 | 11.269 | 15.28 | \-22.01 | 0.08345 |
36
- | IQ2_XS | 2926 | 10.5628 | 16.59 | \-16.8 | 0.07809 |
37
- | IQ2_S | 3063 | 10.3671 | 17.37 | \-15.23 | 0.07772 |
38
- | IQ2_M | 3276 | 9.7973 | 18.58 | \-10.3 | 0.07298 |
39
- | Q2_K_S | 3388 | 9.9206 | 19.21 | \-11.41 | 0.07247 |
40
- | IQ3_XXS | 3621 | 9.3955 | 20.53 | \-6.46 | 0.06962 |
41
- | Q2_K | 3630 | 9.421 | 20.58 | \-6.71 | 0.0683 |
42
- | IQ3_XS | 3953 | 9.2545 | 22.42 | \-5.04 | 0.06868 |
43
- | IQ3_S | 4137 | 9.2127 | 23.46 | \-4.61 | 0.06866 |
44
- | Q3_K_S | 4137 | 9.083 | 23.46 | \-3.24 | 0.06618 |
45
- | IQ3_M | 4287 | 8.9791 | 24.31 | \-2.12 | 0.06614 |
46
- | Q3_K_M | 4542 | 9.0172 | 25.76 | \-2.54 | 0.06684 |
47
- | Q3_K_L | 4895 | 8.9965 | 27.76 | \-2.31 | 0.06675 |
48
- | IQ4_XS | 4943 | 8.8286 | 28.03 | \-0.46 | 0.06504 |
49
- | IQ4_NL | 5191 | 8.8235 | 29.44 | \-0.4 | 0.06496 |
50
- | Q4_0 | 5207 | 8.834 | 29.53 | \-0.52 | 0.0648 |
51
- | Q4_K_S | 5226 | 8.829 | 29.63 | \-0.46 | 0.06513 |
52
- | Q4_K_M | 5495 | 8.8069 | 31.16 | \-0.21 | 0.06493 |
53
- | Q4_1 | 5688 | 8.8395 | 32.25 | \-0.58 | 0.06526 |
54
- | Q5_K_S | 6184 | 8.8011 | 35.07 | \-0.14 | 0.06504 |
55
- | Q5_0 | 6199 | 8.7668 | 35.15 | 0.25 | 0.06455 |
56
- | Q5_K_M | 6340 | 8.7993 | 35.95 | \-0.12 | 0.06506 |
57
- | Q5_1 | 6680 | 8.7888 | 37.88 | 0 | 0.06493 |
58
- | Q6_K | 7238 | 8.7863 | 41.04 | 0.02 | 0.06497 |
59
- | Q8_0 | 9372 | 8.7858 | 53.14 | 0.03 | 0.06497 |
60
- | F16 | 17635 | 8.7884 | 100 | 0 | 0.06501 |
61
 
62
  <hr><br>
63
 
 
30
 
31
  | Quant | Size (MB) | Perplexity (PPL) | Size (%) | Accuracy (%) | PPL Error rate |
32
  | ------- | --------- | ---------------- | -------- | ------------ | -------------- |
33
+ | IQ1_S | 2269 | 16.0064 | 12.87 | 54.91 | 0.12077 |
34
+ | IQ1_M | 2429 | 13.7255 | 13.77 | 64.03 | 0.10272 |
35
+ | IQ2_XXS | 2695 | 11.269 | 15.28 | 77.99 | 0.08345 |
36
+ | IQ2_XS | 2926 | 10.5628 | 16.59 | 83.2 | 0.07809 |
37
+ | IQ2_S | 3063 | 10.3671 | 17.37 | 84.77 | 0.07772 |
38
+ | IQ2_M | 3276 | 9.7973 | 18.58 | 89.7 | 0.07298 |
39
+ | Q2_K_S | 3388 | 9.9206 | 19.21 | 88.59 | 0.07247 |
40
+ | IQ3_XXS | 3621 | 9.3955 | 20.53 | 93.54 | 0.06962 |
41
+ | Q2_K | 3630 | 9.421 | 20.58 | 93.29 | 0.0683 |
42
+ | IQ3_XS | 3953 | 9.2545 | 22.42 | 94.96 | 0.06868 |
43
+ | IQ3_S | 4137 | 9.2127 | 23.46 | 95.39 | 0.06866 |
44
+ | Q3_K_S | 4137 | 9.083 | 23.46 | 96.76 | 0.06618 |
45
+ | IQ3_M | 4287 | 8.9791 | 24.31 | 97.88 | 0.06614 |
46
+ | Q3_K_M | 4542 | 9.0172 | 25.76 | 97.46 | 0.06684 |
47
+ | Q3_K_L | 4895 | 8.9965 | 27.76 | 97.69 | 0.06675 |
48
+ | IQ4_XS | 4943 | 8.8286 | 28.03 | 99.54 | 0.06504 |
49
+ | IQ4_NL | 5191 | 8.8235 | 29.44 | 99.6 | 0.06496 |
50
+ | Q4_0 | 5207 | 8.834 | 29.53 | 99.48 | 0.0648 |
51
+ | Q4_K_S | 5226 | 8.829 | 29.63 | 99.54 | 0.06513 |
52
+ | Q4_K_M | 5495 | 8.8069 | 31.16 | 99.79 | 0.06493 |
53
+ | Q4_1 | 5688 | 8.8395 | 32.25 | 99.42 | 0.06526 |
54
+ | Q5_K_S | 6184 | 8.8011 | 35.07 | 99.86 | 0.06504 |
55
+ | Q5_0 | 6199 | 8.7668 | 35.15 | 100.25 | 0.06455 |
56
+ | Q5_K_M | 6340 | 8.7993 | 35.95 | 99.88 | 0.06506 |
57
+ | Q5_1 | 6680 | 8.7888 | 37.88 | 100 | 0.06493 |
58
+ | Q6_K | 7238 | 8.7863 | 41.04 | 100.02 | 0.06497 |
59
+ | Q8_0 | 9372 | 8.7858 | 53.14 | 100.03 | 0.06497 |
60
+ | F16 | 17635 | 8.7884 | 100 | 100 | 0.06501 |
61
 
62
  <hr><br>
63