metadata
license: mit
This repository aims to explore the extreme compression ratio of the model, so only low bit quantization models are provided. They all quantized from F16.
| model | size | ppl |
|---|---|---|
| F16 | 15G | 8.3662 +/- 0.06216 |
| IQ2_M | 2.8G | 10.2360 +/- 0.07470 |
| IQ2_S | 2.6G | 11.3735 +/- 0.08396 |
| IQ2_XS | 2.5G | 12.3081 +/- 0.08961 |
| IQ2_XXS | 2.3G | 15.9081 +/- 0.11701 |
| IQ1_M | 2.1G | 26.5610 +/- 0.19391 |