Nobody felt like quantizing this model?
#2
by
ElvisM
- opened
Weird. Usually GGUF quants pop out in the first hour.
New architecture, it'll take time for the popular inference engines / quant libs to be updated to support it.
Looks like there's an MLX pull request with support if you have a mac
There are many quantized versions available! -- https://huggingface.co/models?other=base_model:quantized:CohereLabs/c4ai-command-r7b-12-2024
alexrs
changed discussion status to
closed