---
license: cc-by-sa-4.0
---

This repository contains the most relevant quantizations of Stability AI's
[StableLM-3B-4E1T](https://huggingface.co/stabilityai/stablelm-3b-4e1t) model
in GGUF format - ready to be used with
[llama.cpp](https://github.com/ggerganov/llama.cpp) and similar applications.

> For this model, Stability AI claims: "_StableLM-3B-4E1T achieves
> state-of-the-art performance (September 2023) at the 3B parameter scale
> for open-source models and is competitive with many of the popular
> contemporary 7B models, even outperforming our most recent 7B
> StableLM-Base-Alpha-v2._"

Right now, the following quantizations are available:

* [stablelm-3b-4e1t-Q4_K_M](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q4_K_M.bin)
* [stablelm-3b-4e1t-Q5_K_M](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q5_K_M.bin)
* [stablelm-3b-4e1t-Q6_K](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q6_K.bin)
* [stablelm-3b-4e1t-Q8_K](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q8_K.bin)

These files are presented here with the written permission of Stability AI (although
access to the model itself is "gated").

Any model details can be found on the
[original model card](https://huggingface.co/stabilityai/stablelm-3b-4e1t) and in
a paper on [StableLM-3B-4E1T](https://stability.wandb.io/stability-llm/stable-lm/reports/StableLM-3B-4E1T--VmlldzoyMjU4?accessToken=u3zujipenkx5g7rtcj9qojjgxpconyjktjkli2po09nffrffdhhchq045vp0wyfo).
The most important ones are

* context length is 4096


The chosen license is the same as that
of the original model.

(more details will follow a.s.a.p.)