Thank you for EXL2 variants =)
#1
by
xpgx1
- opened
I have sadly no other feedback than to say -> Thank you for converting this model into the new EXL2 variant, even the 12 GB VRAM gang has some enthusiasts craving higher quantized numbers. =) Cheers!
Exciting times indeed.
Yeah, I'm sure quite a few people with obsolete GPUs would appreciate 2 or 3 bit variants of 13b models. I have a 3060, so the 4 and 5 bit variations offered here work fine for me, but I'm salivating over the idea of running a 2-bit frankensteined 22b model. Wherever people happen to fall on the spectrum, it seems these exl2 variants allow them to go way higher than before. Very exciting times!