5.0BPW Quant?
#1
by
Adzeiros
- opened
Is it possible for you to upload a 5.0bpw version?
My script crushed in the middle of the quantization, so that's why only 3bpw and 5.5bpw unfortunately.
You can use the measurement JSON to quant for any bpw you wish.
My cards are currently busy and will be for the next few days. If there won't be any quants in the coming week, and if you can remind me- and I'll gladly do additional quants.
SicariusSicariiStuff
changed discussion status to
closed
Just a friendly reminder :) If not, maybe I will give it a shot. Idk how to quant but I am sure there are videos
Ty for the remind I really did forget about 🥲
Unfortunately, all my cards are busy, but you can definitely do the quant yourself, it's not too complicated, try looking here:
https://github.com/turboderp/exllamav2