Add links to models
Browse files
README.md
CHANGED
|
@@ -29,14 +29,14 @@ The GGUF version allows you to run, among other things, in [LM Studio](https://l
|
|
| 29 |
|
| 30 |
| Filename | Size | Quantization type | Recommended hardware | Usage |
|
| 31 |
|-------------|---------|-----------------|-----------------|--------------|
|
| 32 |
-
| PLLuM-8x7B-chat-gguf-q2_k.gguf | 17 GB | Q2_K | CPU, min. 20 GB RAM | Very weak computers, worst quality |
|
| 33 |
-
| **PLLuM-8x7B-chat-gguf-iq3_s.gguf** | 20.4 GB | IQ3_S | CPU, min. 24GB RAM | Running on weaker computers with acceptable quality |
|
| 34 |
-
| PLLuM-8x7B-chat-gguf-q3_k_m.gguf | 22.5 GB | Q3_K_M | CPU, min. 26GB RAM | Good compromise between size and quality |
|
| 35 |
-
| PLLuM-8x7B-chat-gguf-q4_k_m.gguf | 28.4 GB | Q4_K_M | CPU/GPU, min. 32GB RAM | Recommended for most applications |
|
| 36 |
-
| PLLuM-8x7B-chat-gguf-q5_k_m.gguf | 33.2 GB | Q5_K_M | CPU/GPU, min. 40GB RAM | High quality with reasonable size |
|
| 37 |
-
| PLLuM-8x7B-chat-gguf-q8_0.gguf | 49.6 GB | Q8_0 | GPU, min. 52GB RAM | Highest quality, close to original |
|
| 38 |
-
| **PLLuM-8x7B-chat-gguf-F16** | ~85 GB | F16 | GPU, min. 85GB VRAM | Reference model without quantization |
|
| 39 |
-
| **PLLuM-8x7B-chat-gguf-bf16** | ~85 GB | BF16 | GPU, min. 85GB VRAM | Alternative full precision format |
|
| 40 |
|
| 41 |
## What is quantization?
|
| 42 |
|
|
|
|
| 29 |
|
| 30 |
| Filename | Size | Quantization type | Recommended hardware | Usage |
|
| 31 |
|-------------|---------|-----------------|-----------------|--------------|
|
| 32 |
+
| [PLLuM-8x7B-chat-gguf-q2_k.gguf](https://huggingface.co/piotrmaciejbednarski/PLLuM-8x7B-chat-GGUF/blob/main/PLLuM-8x7B-chat-gguf-q2_k.gguf) | 17 GB | Q2_K | CPU, min. 20 GB RAM | Very weak computers, worst quality |
|
| 33 |
+
| [**PLLuM-8x7B-chat-gguf-iq3_s.gguf**](https://huggingface.co/piotrmaciejbednarski/PLLuM-8x7B-chat-GGUF/blob/main/PLLuM-8x7B-chat-gguf-iq3_s.gguf) | 20.4 GB | IQ3_S | CPU, min. 24GB RAM | Running on weaker computers with acceptable quality |
|
| 34 |
+
| [PLLuM-8x7B-chat-gguf-q3_k_m.gguf](https://huggingface.co/piotrmaciejbednarski/PLLuM-8x7B-chat-GGUF/blob/main/PLLuM-8x7B-chat-gguf-q3_k_m.gguf) | 22.5 GB | Q3_K_M | CPU, min. 26GB RAM | Good compromise between size and quality |
|
| 35 |
+
| [PLLuM-8x7B-chat-gguf-q4_k_m.gguf](https://huggingface.co/piotrmaciejbednarski/PLLuM-8x7B-chat-GGUF/blob/main/PLLuM-8x7B-chat-gguf-q4_k_m.gguf) | 28.4 GB | Q4_K_M | CPU/GPU, min. 32GB RAM | Recommended for most applications |
|
| 36 |
+
| [PLLuM-8x7B-chat-gguf-q5_k_m.gguf](https://huggingface.co/piotrmaciejbednarski/PLLuM-8x7B-chat-GGUF/blob/main/PLLuM-8x7B-chat-gguf-q5_k_m.gguf) | 33.2 GB | Q5_K_M | CPU/GPU, min. 40GB RAM | High quality with reasonable size |
|
| 37 |
+
| [PLLuM-8x7B-chat-gguf-q8_0.gguf](https://huggingface.co/piotrmaciejbednarski/PLLuM-8x7B-chat-GGUF/blob/main/PLLuM-8x7B-chat-gguf-q8_0.gguf) | 49.6 GB | Q8_0 | GPU, min. 52GB RAM | Highest quality, close to original |
|
| 38 |
+
| [**PLLuM-8x7B-chat-gguf-F16**](https://huggingface.co/piotrmaciejbednarski/PLLuM-8x7B-chat-GGUF/tree/main/PLLuM-8x7B-chat-gguf-F16) | ~85 GB | F16 | GPU, min. 85GB VRAM | Reference model without quantization |
|
| 39 |
+
| [**PLLuM-8x7B-chat-gguf-bf16**](https://huggingface.co/piotrmaciejbednarski/PLLuM-8x7B-chat-GGUF/tree/main/PLLuM-8x7B-chat-gguf-bf16) | ~85 GB | BF16 | GPU, min. 85GB VRAM | Alternative full precision format |
|
| 40 |
|
| 41 |
## What is quantization?
|
| 42 |
|