neuralmagic
/

Sparse-Llama-3.1-8B-gsm8k-2of4-quantized.w4a16

Text Generation

compressed-tensors

Model card Files Files and versions Community

nm-research commited on Dec 18, 2024

Commit

1ba4bbc

·

verified ·

1 Parent(s): a9d9e16

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -72,7 +72,7 @@
   },
   "rope_theta": 500000.0,
   "tie_word_embeddings": false,
-  "torch_dtype": "bfloat16",
   "transformers_version": "4.46.2",
   "use_cache": false,
   "vocab_size": 128256

   },
   "rope_theta": 500000.0,
   "tie_word_embeddings": false,
+  "torch_dtype": "float16",
   "transformers_version": "4.46.2",
   "use_cache": false,
   "vocab_size": 128256