Update README.md
#32
by
MaziyarPanahi
- opened
README.md
CHANGED
|
@@ -42,6 +42,13 @@ On April 10th, [@MistralAI](https://huggingface.co/mistralai) released a model n
|
|
| 42 |
|
| 43 |
The GGUF and quantized models here are based on [v2ray/Mixtral-8x22B-v0.1](https://huggingface.co/v2ray/Mixtral-8x22B-v0.1) model
|
| 44 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
## Load sharded model
|
| 46 |
|
| 47 |
`llama_load_model_from_file` will detect the number of files and will load additional tensors from the rest of files.
|
|
|
|
| 42 |
|
| 43 |
The GGUF and quantized models here are based on [v2ray/Mixtral-8x22B-v0.1](https://huggingface.co/v2ray/Mixtral-8x22B-v0.1) model
|
| 44 |
|
| 45 |
+
## How to download
|
| 46 |
+
You can download only the quants you need instead of cloning the entire repository as follows:
|
| 47 |
+
|
| 48 |
+
```
|
| 49 |
+
huggingface-cli download MaziyarPanahi/WizardLM-2-8x22B-GGUF --local-dir . --include '*Q2_K*gguf'
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
## Load sharded model
|
| 53 |
|
| 54 |
`llama_load_model_from_file` will detect the number of files and will load additional tensors from the rest of files.
|