---
license: cc-by-sa-4.0
---

# StableLM-3B-4E1T #

* Model Creator: [Stability AI](https://huggingface.co/stabilityai)
* original Model: [StableLM-3B-4E1T](https://huggingface.co/stabilityai/stablelm-3b-4e1t)

## Description ## 

This repository contains the most relevant quantizations of Stability AI's
[StableLM-3B-4E1T](https://huggingface.co/stabilityai/stablelm-3b-4e1t) model
in GGUF format - ready to be used with
[llama.cpp](https://github.com/ggerganov/llama.cpp) and similar applications.

## About StableLM-3B-4E1T ##

Stability AI claims: "_StableLM-3B-4E1T achieves
state-of-the-art performance (September 2023) at the 3B parameter scale
for open-source models and is competitive with many of the popular
contemporary 7B models, even outperforming our most recent 7B
StableLM-Base-Alpha-v2._"

According to them "_The model is intended to be used as a foundational base 
model for application-specific fine-tuning. Developers must evaluate and 
fine-tune the model for safe performance in downstream applications._"

## Files ##

Right now, the following quantizations are available:

* [stablelm-3b-4e1t-Q4_K_M](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q4_K_M.bin)
* [stablelm-3b-4e1t-Q5_K_M](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q5_K_M.bin)
* [stablelm-3b-4e1t-Q6_K](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q6_K.bin)
* [stablelm-3b-4e1t-Q8_K](https://huggingface.co/rozek/StableLM-3B-4E1T_GGUF/blob/main/stablelm-3b-4e1t-Q8_K.bin)

(tell me if you need more)

These files are presented here with the written permission of Stability AI (although
access to the model itself is still "gated").

## Usage Details ##

Any technical details can be found on the
[original model card](https://huggingface.co/stabilityai/stablelm-3b-4e1t) and in
a paper on [StableLM-3B-4E1T](https://stability.wandb.io/stability-llm/stable-lm/reports/StableLM-3B-4E1T--VmlldzoyMjU4?accessToken=u3zujipenkx5g7rtcj9qojjgxpconyjktjkli2po09nffrffdhhchq045vp0wyfo).
The most important ones for using this model are

* context length is 4096
* there does not seem to be a specific prompt structure - just provide the text
you want to be completed

### Text Completion with LLaMA.cpp ###

For simple inferencing, use a command like 

```
./main -m stablelm-3b-4e1t-Q8_0.bin --temp 0 --top-k 4 --prompt "who was Joseph Weizenbaum?"
```

### Text Tokenization with LLaMA.cpp ###

To get a list of tokens, use a command like 

```
./tokenization -m stablelm-3b-4e1t-Q8_0.bin --prompt "who was Joseph Weizenbaum?"
```

### Embeddings Calculation with LLaMA.cpp ###

Text embeddings are calculated with a command like 

```
./embedding -m stablelm-3b-4e1t-Q8_0.bin --prompt "who was Joseph Weizenbaum?"
```


## License ##

The original "_Model checkpoints are licensed under the Creative Commons license
([CC BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)). Under this 
license, you must give [credit](https://creativecommons.org/licenses/by/4.0/#) 
to Stability AI, provide a link to the license, and 
[indicate if changes were made](https://creativecommons.org/licenses/by/4.0/#). 
You may do so in any reasonable manner, but not in any way that suggests the Stability AI endorses you or your use._"

So, in order to be fair and give credits to whom they belong:

* the original model was created and published by [Stability AI](https://huggingface.co/stabilityai)
* besides quantization, no changes were applied to the model itself