DavidAU commited on
Commit
8a02337
·
verified ·
1 Parent(s): 977f0bb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -38,7 +38,7 @@ pipeline_tag: text-generation
38
  - All quants have been "refreshed", quanted with the lastest LLAMACPP improvements : Better instruction following, output generation across all quants.
39
  - All quants have also been upgraded with "more bits" for output tensor and embed for better performance (this is in addition to the "refresh")
40
  - New "ARM" quants have been added for machines than can run them. (format: ".../Q4_0_4_4.gguf")
41
- - New specialized additional quants (in addition to standard): "max, max-cpu" (will include this in the file name) for quants "Q2K" (max cpu only), "IQ4_XS", "Q6_K" and "Q8_0"
42
  - "MAX": output tensor / embed at float 16. (better instruction following/output generation than standard quants)
43
  - "MAX-CPU": output tensor / embed at bfloat 16, which forces these on to the CPU (Nvidia cards / other will vary), this frees up vram at cost of token/second and you get better instruction following/output generation too.
44
 
 
38
  - All quants have been "refreshed", quanted with the lastest LLAMACPP improvements : Better instruction following, output generation across all quants.
39
  - All quants have also been upgraded with "more bits" for output tensor and embed for better performance (this is in addition to the "refresh")
40
  - New "ARM" quants have been added for machines than can run them. (format: ".../Q4_0_4_4.gguf")
41
+ - New specialized quants (in addition to standard): "max, max-cpu" (will include this in the file name) for quants "Q2K" (max cpu only), "IQ4_XS", "Q6_K" and "Q8_0"
42
  - "MAX": output tensor / embed at float 16. (better instruction following/output generation than standard quants)
43
  - "MAX-CPU": output tensor / embed at bfloat 16, which forces these on to the CPU (Nvidia cards / other will vary), this frees up vram at cost of token/second and you get better instruction following/output generation too.
44