Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- text-generation
|
| 7 |
+
- conversational
|
| 8 |
+
- ggml
|
| 9 |
+
inference: false
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# Metharme 1.3B GGML
|
| 13 |
+
### This repository contains quantized conversions of the Metharme 1.3B checkpoint.
|
| 14 |
+
*For use with frontends that support GGML quantized GPT-NeoX models, such as KoboldCpp and Oobabooga (with the CTransformers loader).*
|
| 15 |
+
|
| 16 |
+
*Last updated on 2023-10-07.*
|
| 17 |
+
|
| 18 |
+
**Description:**
|
| 19 |
+
- The motivation behind these quantizations was seeing Metharme 1.3B outscore Pygmalion 1.3B on the Open LLM Leaderboard, and having a more modern prompt template. Fun fact: ["Pygmalion 2"](https://huggingface.co/PygmalionAI/pygmalion-2-7b) is part of the Metharme series.
|
| 20 |
+
|
| 21 |
+
Model | Startup RAM usage (KoboldCpp) | Startup RAM usage (Oobabooga)
|
| 22 |
+
:--:|:--:|:--:
|
| 23 |
+
metharme-1.3b.q4_0.bin | 1.0 GiB | 1.3 GiB
|
| 24 |
+
metharme-1.3b.q4_1.bin | 1.1 GiB | 1.4 GiB
|
| 25 |
+
metharme-1.3b.q5_0.bin | 1.2 GiB | 1.5 GiB
|
| 26 |
+
metharme-1.3b.q5_1.bin | 1.3 GiB | 1.6 GiB
|
| 27 |
+
metharme-1.3b.q8_0.bin | 1.7 GiB | 2.0 GiB
|
| 28 |
+
metharme-1.3b.f16.bin | 2.9 GiB | 3.2 GiB
|