This is 4-bit quantization of chavinlo/alpaca-native (cecc16d
) via qwopqwop200/GPTQ-for-LLaMa (5cdfad2
).
Invoked as:
llama.py /output/path c4 --wbits 4 --groupsize 128 --save alpaca7b-4bit.pt
This is 4-bit quantization of chavinlo/alpaca-native (cecc16d
) via qwopqwop200/GPTQ-for-LLaMa (5cdfad2
).
Invoked as:
llama.py /output/path c4 --wbits 4 --groupsize 128 --save alpaca7b-4bit.pt