boxomcfoxo commited on
Commit
3a4adef
·
verified ·
1 Parent(s): 15038ca

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -7
README.md CHANGED
@@ -49,8 +49,7 @@ Below is an instruction that describes a task. Write a response that appropriate
49
  <!-- recommended-settings start -->
50
  ## Recommended settings
51
 
52
- - Default preset if using SillyTavern.
53
-
54
  - Temperature: 0.7
55
  - Min-P: 0.3
56
  - Amount to generate: 256
@@ -72,11 +71,11 @@ The Apache 2.0 license requires that attribution is included at the point of rel
72
  <details>
73
  <summary>Click to see details</summary>
74
  The new methods available are:
75
- * GGML_TYPE_Q2_K - "type-1" 2-bit quantization in super-blocks containing 16 blocks, each block having 16 weight. Block scales and mins are quantized with 4 bits. This ends up effectively using 2.5625 bits per weight (bpw)
76
- * GGML_TYPE_Q3_K - "type-0" 3-bit quantization in super-blocks containing 16 blocks, each block having 16 weights. Scales are quantized with 6 bits. This end up using 3.4375 bpw.
77
- * GGML_TYPE_Q4_K - "type-1" 4-bit quantization in super-blocks containing 8 blocks, each block having 32 weights. Scales and mins are quantized with 6 bits. This ends up using 4.5 bpw.
78
- * GGML_TYPE_Q5_K - "type-1" 5-bit quantization. Same super-block structure as GGML_TYPE_Q4_K resulting in 5.5 bpw
79
- * GGML_TYPE_Q6_K - "type-0" 6-bit quantization. Super-blocks with 16 blocks, each block having 16 weights. Scales are quantized with 8 bits. This ends up using 6.5625 bpw
80
  Refer to the Provided Files table below to see what files use which methods, and how.
81
  </details>
82
  <!-- quantization_methods end -->
 
49
  <!-- recommended-settings start -->
50
  ## Recommended settings
51
 
52
+ - Default preset if using SillyTavern
 
53
  - Temperature: 0.7
54
  - Min-P: 0.3
55
  - Amount to generate: 256
 
71
  <details>
72
  <summary>Click to see details</summary>
73
  The new methods available are:
74
+ - GGML_TYPE_Q2_K - "type-1" 2-bit quantization in super-blocks containing 16 blocks, each block having 16 weight. Block scales and mins are quantized with 4 bits. This ends up effectively using 2.5625 bits per weight (bpw)
75
+ - GGML_TYPE_Q3_K - "type-0" 3-bit quantization in super-blocks containing 16 blocks, each block having 16 weights. Scales are quantized with 6 bits. This end up using 3.4375 bpw.
76
+ - GGML_TYPE_Q4_K - "type-1" 4-bit quantization in super-blocks containing 8 blocks, each block having 32 weights. Scales and mins are quantized with 6 bits. This ends up using 4.5 bpw.
77
+ - GGML_TYPE_Q5_K - "type-1" 5-bit quantization. Same super-block structure as GGML_TYPE_Q4_K resulting in 5.5 bpw
78
+ - GGML_TYPE_Q6_K - "type-0" 6-bit quantization. Super-blocks with 16 blocks, each block having 16 weights. Scales are quantized with 8 bits. This ends up using 6.5625 bpw
79
  Refer to the Provided Files table below to see what files use which methods, and how.
80
  </details>
81
  <!-- quantization_methods end -->