D_AU - Experiments in Merging Top Models
Collection
Full precision and Q8 of merged models by me. Focus in this collection is creative, role play, story and fiction. Imatrix GGUFs addition(s) pending.
β’
48 items
β’
Updated
β’
2
Experimental: Tiefighter Expanded to 15.6 B / 48 layers.
D_AU-Tiefighter-Expanded-13B-pass is a merge of the following models using LazyMergekit:
slices:
- sources:
- model: KoboldAI/LLaMA2-13B-Tiefighter
layer_range: [0, 40]
- sources:
- model: KoboldAI/LLaMA2-13B-Tiefighter
layer_range: [32, 40]
merge_method: passthrough
dtype: bfloat16
!pip install -qU transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "DavidAU/D_AU-Tiefighter-Expanded-13B-pass"
messages = [{"role": "user", "content": "What is a large language model?"}]
tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
Base model
KoboldAI/LLaMA2-13B-Tiefighter