Can this be quantized/Are there quantized variants available?
#16
by
popeyed
- opened
This is very useful. Thank you for making it. But it's very large for 3.3B since it's the full model. I would love to know if quantization is possible/whether it affects the quality too much for this type of models.