ppx info
Browse files
README.md
CHANGED
|
@@ -17,3 +17,14 @@ Support is now merged to master branch.
|
|
| 17 |
|
| 18 |
There are now more quantization types in llama.cpp, some lower than 4 bits.
|
| 19 |
Currently these are not supported, maybe because some weights have shapes that don't divide by 256.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
There are now more quantization types in llama.cpp, some lower than 4 bits.
|
| 19 |
Currently these are not supported, maybe because some weights have shapes that don't divide by 256.
|
| 20 |
+
|
| 21 |
+
## Perplexity on wiki.test.raw
|
| 22 |
+
|
| 23 |
+
| Q | chunk | 600BT | 1000BT |
|
| 24 |
+
| --: | --: | --: | --: |
|
| 25 |
+
| F16 | [616] | 8.4656 | 7.7861 |
|
| 26 |
+
| Q8_0 | [616] | 8.4667 | 7.7874 |
|
| 27 |
+
| Q5_1 | [616] | 8.5072 | 7.8424 |
|
| 28 |
+
| Q5_0 | [616] | 8.5156 | 7.8474 |
|
| 29 |
+
| Q4_1 | [616] | 8.6102 | 8.0483 |
|
| 30 |
+
| Q4_0 | [616] | 8.6674 | 8.0962 |
|