Moxoff/Mistral-Ita

Unique Features for Italian

• Tailored Vocabulary: The model's vocabulary is fine-tuned to encompass the nuances and diversity of the Italian language.
• Enhanced Understanding: Mistral-7B is specifically trained to grasp and generate Italian text, ensuring high linguistic and contextual accuracy.

4-Bit Quantized Model Download

The model quantized to 4 bits is available for download at this link: mistal-Ita-4bit.gguf

How to Use

How to utilize my Mistral for Italian text generation

import transformers
from transformers import TextStreamer
import torch

model_name = "Moxoff/Mistral-Ita"

tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)
model = transformers.LlamaForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto").eval()

def stream(user_prompt):
    runtimeFlag = "cuda:0"
    system_prompt = ''
    B_INST, E_INST = "[INST]", "[/INST]"
    prompt = f"{system_prompt}{B_INST}{user_prompt.strip()}\n{E_INST}"
    inputs = tokenizer([prompt], return_tensors="pt").to(runtimeFlag)
    streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
    _ = model.generate(**inputs, streamer=streamer, max_new_tokens=100, num_return_sequences=1)

domanda = """Scrivi una funzione python che calcola la media di questi numeri"""
contesto = """
[-5, 10, 15, 20, 25, 30, 35]
"""

prompt = domanda + "\n" + contesto

stream(prompt)