|
--- |
|
language: |
|
- it |
|
--- |
|
|
|
## Unique Features for Italian |
|
- **Tailored Vocabulary**: The model's vocabulary is fine-tuned to encompass the nuances and diversity of the Italian language. |
|
- **Enhanced Understanding**: Mistral-7B is specifically trained to grasp and generate Italian text, ensuring high linguistic and contextual accuracy. |
|
|
|
## 4-Bit Quantized Model Download |
|
The model quantized to 4 bits is available for download at this link: [mistal-Ita-4bit.gguf](https://huggingface.co/Moxoff/Mistral-Ita-4bit/blob/main/mistal-Ita-Q4_K_M.gguf) |
|
## How to Use |
|
How to utilize my Mistral for Italian text generation |
|
|
|
```python |
|
import transformers |
|
from transformers import TextStreamer |
|
import torch |
|
|
|
model_name = "Moxoff/Mistral-Ita" |
|
|
|
tokenizer = transformers.AutoTokenizer.from_pretrained(model_name) |
|
model = transformers.LlamaForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, device_map="auto").eval() |
|
|
|
def stream(user_prompt): |
|
runtimeFlag = "cuda:0" |
|
system_prompt = '' |
|
B_INST, E_INST = "[INST]", "[/INST]" |
|
prompt = f"{system_prompt}{B_INST}{user_prompt.strip()}\n{E_INST}" |
|
inputs = tokenizer([prompt], return_tensors="pt").to(runtimeFlag) |
|
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True) |
|
_ = model.generate(**inputs, streamer=streamer, max_new_tokens=100, num_return_sequences=1) |
|
|
|
domanda = """Scrivi una funzione python che calcola la media di questi numeri""" |
|
contesto = """ |
|
[-5, 10, 15, 20, 25, 30, 35] |
|
""" |
|
|
|
prompt = domanda + "\n" + contesto |
|
|
|
stream(prompt) |
|
``` |
