OPEA
/

DeepSeek-V3-int4-sym-awq-inc-cpu

4-bit precision

Model card Files Files and versions Community

wenhuach commited on 20 days ago

Commit

c758202

·

1 Parent(s): 16eb0b2

awq format

Signed-off-by: wenhuach <[email protected]>

Files changed (1) hide show

config.json +1 -2

config.json CHANGED Viewed

@@ -57,8 +57,7 @@
       "lm_head"
     ],
     "nsamples": 512,
-    "backend": "auto_round:awq:gemm",
-    "quant_method": "intel/auto-round",
     "scale_dtype": "torch.float16",
     "seqlen": 2048,
     "sym": true,

       "lm_head"
     ],
     "nsamples": 512,
+    "quant_method": "awq",
     "scale_dtype": "torch.float16",
     "seqlen": 2048,
     "sym": true,