etemiz
/

Llama-3.1-405B-Inst-GGUF

Inference Endpoints

Model card Files Files and versions Community

etemiz commited on Jul 28

Commit

5d5fbdc

•

1 Parent(s): e0b0d57

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -4,6 +4,7 @@ license: llama3.1
 Llama 3.1 405B Quants
 - IQ1_S: 86.8 GB
 - IQ1_M: 95.1 GB
 Quantization from BF16 here:
 https://huggingface.co/nisten/meta-405b-instruct-cpu-optimized-gguf/

 Llama 3.1 405B Quants
 - IQ1_S: 86.8 GB
 - IQ1_M: 95.1 GB
+- IQ2_XXS: 109.0 GB
 Quantization from BF16 here:
 https://huggingface.co/nisten/meta-405b-instruct-cpu-optimized-gguf/