Hugging Face's logo

CCRss
/
Llama-3.1-Nemotron-70B-Instruct-HF-calib-original_pileval_en_AWQ-4bit-g128-gemm

Safetensors
llama
4-bit precision
awq
Model card Files Community
CCRss commited on
Commit
edcff9d
·
verified ·
1 Parent(s): f75f20c

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ quant_config = {
2
+ "zero_point": True,
3
+ "q_group_size": 128,
4
+ "w_bit": 4,
5
+ "version": "GEMM"
6
+ }