perplexity.md

Quant	Size (MB)	PPL	Size (%)	Accuracy (%)	PPL error rate
IQ1_S	14632	12.2904	10.87	44.38	0.19
IQ1_M	15975	10.3993	11.87	52.44	0.15
IQ2_XXS	18212	8.5568	13.53	63.74	0.13
IQ2_XS	20162	7.7272	14.98	70.58	0.11
IQ2_S	21211	7.3268	15.76	74.44	0.10
IQ2_M	23001	6.8673	17.09	79.42	0.10
Q2_K_S	23338	7.2922	17.34	74.79	0.11
Q2_K	25153	6.7633	18.69	80.64	0.10
IQ3_XXS	26196	6.1197	19.47	89.12	0.09
IQ3_XS	27950	5.9414	20.77	91.79	0.08
IQ3_S	29480	5.8043	21.91	93.96	0.08
Q3_K_S	29480	5.9948	21.91	90.98	0.08
IQ3_M	30457	5.7853	22.63	94.27	0.08
Q3_K_M	32680	5.8430	24.28	93.34	0.08
Q3_K_L	35420	5.8050	26.32	93.95	0.08
IQ4_XS	36146	5.6062	26.86	97.28	0.08
IQ4_NL	38198	5.6022	28.38	97.35	0.08
Q4_0	38258	5.8337	28.43	93.49	0.08
Q4_K_S	38478	5.5814	28.59	97.71	0.08
Q4_K_M	40550	5.5582	30.13	98.12	0.08
Q4_1	42260	5.5984	31.40	97.42	0.08
Q5_K_S	46403	5.5143	34.48	98.90	0.08
Q5_0	46543	5.5182	34.58	98.83	0.08
Q5_K_M	47635	5.4842	35.40	99.45	0.08
Q5_1	50545	5.5105	37.56	98.97	0.08
Q6_K	55206	5.4699	41.02	99.71	0.08
Q8_0	71501	5.4532	53.13	100.01	0.08
F16	134580	5.4539	100	100	0.08