Hugging Face's logo

liuhaotian
/
llava-v1.6-34b

Image-Text-to-Text
Safetensors
LLaVA
llava
conversational
Model card Files Community
21

Quantizations?

#1
by musicurgy - opened
Discussion
musicurgy

Would love to see some quantizations of this model.

YaTharThShaRma999

@musicurgy it can be probably quanted by bnb but not gguf or gptq yet.

mobicham

@musicurgy you can quantize it using HQQ: https://github.com/mobiusml/hqq/?tab=readme-ov-file#multimodal

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment