danielhanchen
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -18,7 +18,8 @@ tags:
|
|
18 |
Or you can view more detailed instructions here: [unsloth.ai/blog/deepseekr1-dynamic](https://unsloth.ai/blog/deepseekr1-dynamic)
|
19 |
1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
|
20 |
2. Obtain the latest `llama.cpp` at https://github.com/ggerganov/llama.cpp
|
21 |
-
3.
|
|
|
22 |
```bash
|
23 |
./llama.cpp/llama-cli \
|
24 |
--model DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \
|
|
|
18 |
Or you can view more detailed instructions here: [unsloth.ai/blog/deepseekr1-dynamic](https://unsloth.ai/blog/deepseekr1-dynamic)
|
19 |
1. Do not forget about `<|User|>` and `<|Assistant|>` tokens! - Or use a chat template formatter
|
20 |
2. Obtain the latest `llama.cpp` at https://github.com/ggerganov/llama.cpp
|
21 |
+
3. It's best to use `--min-p 0.05 or 0.1` to counteract very rare token predictions - I found this to work well especially for the 1.58bit model.
|
22 |
+
4. Example with Q4_0 K quantized cache **Notice -no-cnv disables auto conversation mode**
|
23 |
```bash
|
24 |
./llama.cpp/llama-cli \
|
25 |
--model DeepSeek-R1-UD-IQ1_S/DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf \
|