TheYuriLover
commited on
Commit
·
995700b
1
Parent(s):
5b83e62
Update README.md
Browse files
README.md
CHANGED
@@ -2,4 +2,6 @@ This is the gptq 4bit quantization of this model: https://huggingface.co/openacc
|
|
2 |
|
3 |
This quantization was made by using this repository: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton
|
4 |
|
5 |
-
And I used the triton branch with all the gptq implementations available (true_sequential + act_order + groupsize 128)
|
|
|
|
|
|
2 |
|
3 |
This quantization was made by using this repository: https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/triton
|
4 |
|
5 |
+
And I used the triton branch with all the gptq implementations available (true_sequential + act_order + groupsize 128)
|
6 |
+
|
7 |
+
CUDA_VISIBLE_DEVICES=0 python llama.py ./Manticore-13b-GPTQ-Triton c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors Manticore-13b-GPTQ-Triton.safetensors
|