tags: | |
- merge | |
- mergekit | |
- lazymergekit | |
# Aura_3B | |
Aura_3B is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing): | |
## 🧩 Configuration | |
```yaml | |
models: | |
- model: jeiku/Kielbasa_3B+jeiku/Everything_v3_128_StableLM | |
- model: euclaise/ReMask-3B+jeiku/Futa_Erotica_StableLM | |
- model: jeiku/ToxicNoRobotsRosaHermesBoros_3B+jeiku/Gnosis_256_StableLM | |
- model: euclaise/Memphis-scribe-3B+jeiku/Humiliation_StableLM | |
merge_method: model_stock | |
base_model: jeiku/Rosa_v1_3B | |
dtype: float16 | |
``` | |
## 💻 Usage | |
```python | |
!pip install -qU transformers accelerate | |
from transformers import AutoTokenizer | |
import transformers | |
import torch | |
model = "jeiku/Aura_3B" | |
messages = [{"role": "user", "content": "What is a large language model?"}] | |
tokenizer = AutoTokenizer.from_pretrained(model) | |
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) | |
pipeline = transformers.pipeline( | |
"text-generation", | |
model=model, | |
torch_dtype=torch.float16, | |
device_map="auto", | |
) | |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) | |
print(outputs[0]["generated_text"]) | |
``` |