File size: 2,370 Bytes
ed38ae9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
| ------- | --------- | --------- | -------- | ------------ | -------------- |
| IQ1_S | 14632 | 12.2904 | 10.87 | 44.38 | 0.19 |
| IQ1_M | 15975 | 10.3993 | 11.87 | 52.44 | 0.15 |
| IQ2_XXS | 18212 | 8.5568 | 13.53 | 63.74 | 0.13 |
| IQ2_XS | 20162 | 7.7272 | 14.98 | 70.58 | 0.11 |
| IQ2_S | 21211 | 7.3268 | 15.76 | 74.44 | 0.10 |
| IQ2_M | 23001 | 6.8673 | 17.09 | 79.42 | 0.10 |
| Q2_K_S | 23338 | 7.2922 | 17.34 | 74.79 | 0.11 |
| Q2_K | 25153 | 6.7633 | 18.69 | 80.64 | 0.10 |
| IQ3_XXS | 26196 | 6.1197 | 19.47 | 89.12 | 0.09 |
| IQ3_XS | 27950 | 5.9414 | 20.77 | 91.79 | 0.08 |
| IQ3_S | 29480 | 5.8043 | 21.91 | 93.96 | 0.08 |
| Q3_K_S | 29480 | 5.9948 | 21.91 | 90.98 | 0.08 |
| IQ3_M | 30457 | 5.7853 | 22.63 | 94.27 | 0.08 |
| Q3_K_M | 32680 | 5.8430 | 24.28 | 93.34 | 0.08 |
| Q3_K_L | 35420 | 5.8050 | 26.32 | 93.95 | 0.08 |
| IQ4_XS | 36146 | 5.6062 | 26.86 | 97.28 | 0.08 |
| IQ4_NL | 38198 | 5.6022 | 28.38 | 97.35 | 0.08 |
| Q4_0 | 38258 | 5.8337 | 28.43 | 93.49 | 0.08 |
| Q4_K_S | 38478 | 5.5814 | 28.59 | 97.71 | 0.08 |
| Q4_K_M | 40550 | 5.5582 | 30.13 | 98.12 | 0.08 |
| Q4_1 | 42260 | 5.5984 | 31.40 | 97.42 | 0.08 |
| Q5_K_S | 46403 | 5.5143 | 34.48 | 98.90 | 0.08 |
| Q5_0 | 46543 | 5.5182 | 34.58 | 98.83 | 0.08 |
| Q5_K_M | 47635 | 5.4842 | 35.40 | 99.45 | 0.08 |
| Q5_1 | 50545 | 5.5105 | 37.56 | 98.97 | 0.08 |
| Q6_K | 55206 | 5.4699 | 41.02 | 99.71 | 0.08 |
| Q8_0 | 71501 | 5.4532 | 53.13 | 100.01 | 0.08 |
| F16 | 134580 | 5.4539 | 100 | 100 | 0.08 |
|