BlackSamorez's picture
Update README.md
ea1c68f verified
|
raw
history blame
740 Bytes

Official AQLM quantization of mistralai/Mixtral-8x7B-Instruct-v0.1.

For this quantization, we used 1 codebook of 16 bits.

Selected evaluation results for this model:

Model AQLM scheme WinoGrande PiQA HellaSwag ArcE ArcC Model size, Gb Hub link
Mixtral-8x7B-Instruct-v0.1 (THIS) 1x16 0.7593 0.8043 0.6179 0.7768 0.4793 12.6 Link

To learn more about the inference, as well as the information on how to quantize models yourself, please refer to the official GitHub repo.