Update README.md
Browse files
README.md
CHANGED
|
@@ -16,7 +16,24 @@ datasets:
|
|
| 16 |
|
| 17 |
This is [BlinkDL/rwkv-4-pileplus](https://huggingface.co/BlinkDL/rwkv-4-pileplus) converted to GGML for use with rwkv.cpp and KoboldCpp. [rwkv.cpp's conversion instructions](https://github.com/saharNooby/rwkv.cpp#option-32-convert-and-quantize-pytorch-model) were followed.
|
| 18 |
|
| 19 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
* * *
|
| 22 |
|
|
|
|
| 16 |
|
| 17 |
This is [BlinkDL/rwkv-4-pileplus](https://huggingface.co/BlinkDL/rwkv-4-pileplus) converted to GGML for use with rwkv.cpp and KoboldCpp. [rwkv.cpp's conversion instructions](https://github.com/saharNooby/rwkv.cpp#option-32-convert-and-quantize-pytorch-model) were followed.
|
| 18 |
|
| 19 |
+
### RAM USAGE (KoboldCpp)
|
| 20 |
+
Model | RAM usage (with OpenBLAS)
|
| 21 |
+
:--:|:--:
|
| 22 |
+
Unloaded | 41.3 MiB
|
| 23 |
+
169M q4_0 | 249.0 MiB
|
| 24 |
+
169M q5_0 | 254.2 MiB
|
| 25 |
+
169M q5_1 | 259.6 MiB
|
| 26 |
+
430M q4_0 | 443.7 MiB
|
| 27 |
+
430M q5_0 | 463.2 MiB
|
| 28 |
+
430M q5_1 | 482.8 MiB
|
| 29 |
+
1.5B q4_0 | 1.2 GiB
|
| 30 |
+
1.5B q5_0 | 1.3 GiB
|
| 31 |
+
1.5B q5_1 | 1.4 GiB
|
| 32 |
+
3B q4_0 | 2.1 GiB
|
| 33 |
+
3B q5_0 | 2.3 GiB
|
| 34 |
+
3B q5_1 | 2.5 GiB
|
| 35 |
+
|
| 36 |
+
Original model card by BlinkDL is below.
|
| 37 |
|
| 38 |
* * *
|
| 39 |
|