Hugging Face's logo

Quant-Cartel
/
Mistral-Small-NovusKyver-iMat-GGUF

Text Generation
GGUF
Not-For-All-Audiences
iMat
GGUF
Inference Endpoints
imatrix
conversational
Model card Files Community
InferenceIllusionist commited on
Commit
59d5a05
·
verified ·
1 Parent(s): c65ef49

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -27,11 +27,18 @@ C8888 "8" 888 888 " d88888 d88 88b 888
27
  PROUDLY PRESENTS
28
  ```
29
  # Mistral-Small-NovusKyver-iMat-GGUF
 
 
 
 
30
  Quantized with love from fp32.
31
 
32
  Original model author: [envoid](https://huggingface.co/Envoid/)
33
 
34
- * Importance Matrix calculated using [groups_merged.txt](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384) in 105 chunks, n_ctx=512, and fp32 precision weights
 
 
 
35
 
36
  Original model README [here](https://huggingface.co/Envoid/Mistral-Small-NovusKyver/) and below:
37
 
 
27
  PROUDLY PRESENTS
28
  ```
29
  # Mistral-Small-NovusKyver-iMat-GGUF
30
+
31
+ >[!TIP]
32
+ > <b>Quantization Note:</b> For smaller sizes (i.e. IQ3 and below) a repetition penalty of 1.05-1.15 is recommended.
33
+
34
  Quantized with love from fp32.
35
 
36
  Original model author: [envoid](https://huggingface.co/Envoid/)
37
 
38
+ * Importance Matrix calculated using [groups_merged.txt](https://github.com/ggerganov/llama.cpp/discussions/5263#discussioncomment-8395384)
39
+ * 105 chunks
40
+ * n_ctx=512
41
+ * Calculation uses fp32 precision model weights
42
 
43
  Original model README [here](https://huggingface.co/Envoid/Mistral-Small-NovusKyver/) and below:
44