Quantization for NQLSG-Qwen2.5-14B-MegaFusion-v3
Dear Mradermacher,
I have created a specially optimized dynamic quantization version for the model Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v3.
You may proceed with creating the regular static quantization version for this model as well.
Best regards,
Lun Zima
Dear Lun Zima
I queued NQLSG-Qwen2.5-14B-MegaFusion-v3 as requested. Thanks a lot for creating this awesome model. I'm quite impressed by https://huggingface.co/Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v3/discussions/1 so I queued the generation of booth static and imatrix quants. This doesn't prevent anyone from using your for this model optimized and likely much better imatrix quants from https://huggingface.co/Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v3-GGUF
You can check for progress at http://hf.tst.eu/status.html or regularly check the model
summary page at https://hf.tst.eu/model#NQLSG-Qwen2.5-14B-MegaFusion-v3-GGUF for quants to appear.
Best wishes
Nico Bosshard
Dear Nico Bosshard,
According to the discussion thread at https://huggingface.co/Lunzima/NQLSG-Qwen2.5-14B-MegaFusion-v3-GGUF/discussions/1, the dynamic quantization GGUF model aims to provide better performance than Q3_K_L at a similar size, based on the llama.cpp patch mentioned therein.
Best regards,
Lun Zima