3.75 Fits 3x3090 with at least 16k. 3.5 is worse perplexity and 4.0 is slightly too large. Any chance?
Sure. I'll get this posted in a couple hours.
Quant is now up at https://huggingface.co/Dracones/WizardLM-2-8x22B_exl2_3.75bpw
· Sign up or log in to comment