metadata
license: apache-2.0
language:
- en
tags:
- text-generation
- conversational
- ggml
inference: false
Metharme 1.3B GGML
This repository contains quantized conversions of the Metharme 1.3B checkpoint.
For use with frontends that support GGML quantized GPT-NeoX models, such as KoboldCpp and Oobabooga (with the CTransformers loader).
Last updated on 2023-10-07.
Description:
- The motivation behind these quantizations was seeing Metharme 1.3B outscore Pygmalion 1.3B on the Open LLM Leaderboard, and having a more modern prompt template. Fun fact: "Pygmalion 2" is part of the Metharme series.
| Model | Startup RAM usage (KoboldCpp) | Startup RAM usage (Oobabooga) |
|---|---|---|
| metharme-1.3b.q4_0.bin | 1.0 GiB | 1.3 GiB |
| metharme-1.3b.q4_1.bin | 1.1 GiB | 1.4 GiB |
| metharme-1.3b.q5_0.bin | 1.2 GiB | 1.5 GiB |
| metharme-1.3b.q5_1.bin | 1.3 GiB | 1.6 GiB |
| metharme-1.3b.q8_0.bin | 1.7 GiB | 2.0 GiB |
| metharme-1.3b.f16.bin | 2.9 GiB | 3.2 GiB |