zgce
/

acsr-y34b-4bpw-hb6-exl2

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

zgce commited on Nov 16, 2023

Commit

5b718b2

•

1 Parent(s): e4f8711

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -14,7 +14,7 @@ license: mit
 ### description
-- This is test for [exllamav2](https://github.com/turboderp/exllamav2) model version must after [Add Yi support](https://github.com/turboderp/exllamav2/commit/6d24e1ad40d89f64b1bd3ae36e639c74c9f730b2)
 - 4bpw `python convert.py -i acsr-y34b -c exl2/0000.parquet -o acsr-y34b-4bpw-hb6-exl2 -hb 6 -l 4096 -b 4.15`
 - [convert doc](https://github.com/turboderp/exllamav2/blob/master/doc/convert.md)
 - calibration dataset: [WikiText-2-v1](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet)

 ### description
+- This is test for [exllamav2](https://github.com/turboderp/exllamav2) model.
 - 4bpw `python convert.py -i acsr-y34b -c exl2/0000.parquet -o acsr-y34b-4bpw-hb6-exl2 -hb 6 -l 4096 -b 4.15`
 - [convert doc](https://github.com/turboderp/exllamav2/blob/master/doc/convert.md)
 - calibration dataset: [WikiText-2-v1](https://huggingface.co/datasets/wikitext/blob/refs%2Fconvert%2Fparquet/wikitext-2-v1/test/0000.parquet)