Alright I was way too tired lol
Browse files
README.md
CHANGED
|
@@ -14,12 +14,24 @@ datasets:
|
|
| 14 |
|
| 15 |
**Last updated:** 2023-06-07
|
| 16 |
|
| 17 |
-
This is [BlinkDL/rwkv-4-
|
| 18 |
|
| 19 |
### RAM USAGE (KoboldCpp)
|
| 20 |
Model | RAM usage (with OpenBLAS)
|
| 21 |
:--:|:--:
|
| 22 |
Unloaded | 41.3 MiB
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
Original model card by BlinkDL is below.
|
| 25 |
|
|
|
|
| 14 |
|
| 15 |
**Last updated:** 2023-06-07
|
| 16 |
|
| 17 |
+
This is [BlinkDL/rwkv-4-pileplus](https://huggingface.co/BlinkDL/rwkv-4-pileplus) converted to GGML for use with rwkv.cpp and KoboldCpp. [rwkv.cpp's conversion instructions](https://github.com/saharNooby/rwkv.cpp#option-32-convert-and-quantize-pytorch-model) were followed.
|
| 18 |
|
| 19 |
### RAM USAGE (KoboldCpp)
|
| 20 |
Model | RAM usage (with OpenBLAS)
|
| 21 |
:--:|:--:
|
| 22 |
Unloaded | 41.3 MiB
|
| 23 |
+
169M q4_0 | 232.2 MiB
|
| 24 |
+
169M q5_0 | 243.3 MiB
|
| 25 |
+
169M q5_1 | 249.2 MiB
|
| 26 |
+
430M q4_0 | 413.2 MiB
|
| 27 |
+
430M q5_0 | 454.4 MiB
|
| 28 |
+
430M q5_1 | 471.8 MiB
|
| 29 |
+
1.5B q4_0 | 1.1 GiB
|
| 30 |
+
1.5B q5_0 | 1.3 GiB
|
| 31 |
+
1.5B q5_1 | 1.3 GiB
|
| 32 |
+
3B q4_0 | 2.0 GiB
|
| 33 |
+
3B q5_0 | 2.3 GiB
|
| 34 |
+
3B q5_1 | 2.4 GiB
|
| 35 |
|
| 36 |
Original model card by BlinkDL is below.
|
| 37 |
|