Weni
/

ZeroShot-Agents-Llama3-4.0.64-DPO-AWQ

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

ZeroShot-Agents-Llama3-4.0.64-DPO-AWQ / quant_config.json

ironrock's picture

Upload quant_config.json with huggingface_hub

17e92f7 verified 6 months ago

127 Bytes

{"quant_method": "awq", "zero_point": true, "q_group_size": 128, "w_bit": 4, "version": "gemm", "modules_to_not_convert": null}