---
license: apache-2.0
language:
- it
- en
library_name: transformers
tags:
- sft
- it
- mistral
- chatml
---

# Model Information

XXXXQuantized is a compact iteration of the model [XXXX](https://huggingface.co/MoxoffSpA/xxxx), optimized for efficiency.

It is offered in two distinct configurations: a 4-bit version and an 8-bit version, each designed to maintain the model's effectiveness while significantly reducing its size
and computational requirements.

- It's trained both on publicly available datasets, like [SQUAD-it](https://huggingface.co/datasets/squad_it), and datasets we've created in-house.
- it's designed to understand and maintain context, making it ideal for Retrieval Augmented Generation (RAG) tasks and applications requiring contextual awareness.
- It is quantized in a 4-bit version and an 8-bit version folllowing the procedure [here](https://github.com/ggerganov/llama.cpp).

# Evaluation

We evaluated the model using the same test sets as used for the [Open Ita LLM Leaderboard](https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard):

| hellaswag_it acc_norm | arc_it acc_norm | m_mmlu_it 5-shot acc | Average |
|:----------------------| :--------------- | :-------------------- | :------- |
| 0.6067 | 0.4405 | 0.5112 | 0,52 |
| 0.6067 | 0.4405 | 0.5112 | 0,52 |
| 0.6067 | 0.4405 | 0.5112 | 0,52 |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

device = "cuda"

model = AutoModelForCausalLM.from_pretrained("MoxoffSpA/xxxx")
tokenizer = AutoTokenizer.from_pretrained("MoxoffSpA/xxxx")

messages = [
    {"role": "user", "content": "Qual è il tuo piatto preferito??"},
    {"role": "assistant", "content": "Beh, ho un debole per una buona porzione di risotto allo zafferano. È un piatto che si distingue per il suo sapore ricco e il suo bellissimo colore dorato, rendendolo irresistibile!"},
    {"role": "user", "content": "Hai delle ricette con il risotto che consigli?"}
]

encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")

model_inputs = encodeds.to(device)
model.to(device)

generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])
```

## Bias, Risks and Limitations

xxxxQuantized and its original model [xxxx](https://huggingface.co/MoxoffSpA/xxxx) has not been aligned to human preferences for safety within the RLHF phase or deployed with in-the-loop filtering of 
responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition 
of the corpus was used to train the base model [mistralai/Mistral-7B-v0.2](https://huggingface.co/mistralai/Mistral-7B-v0.2), however it is likely to have included a mix of Web data and technical sources 
like books and code.

## Links to resources

- SQUAD-it dataset: https://huggingface.co/datasets/squad_it
- Mistral_7B_v0.2 original weights: https://models.mistralcdn.com/mistral-7b-v0-2/mistral-7B-v0.2.tar
- Mistral_7B_v0.2 model: https://huggingface.co/alpindale/Mistral-7B-v0.2-hf
- Open Ita LLM Leaderbord: https://huggingface.co/spaces/FinancialSupport/open_ita_llm_leaderboard

## Base version

We have the not quantized version here:
https://huggingface.co/MoxoffSpA/xxxx

## The Moxoff Team

Jacopo Abate, Marco D'Ambra, Gianpaolo Francesco Trotta, Luigi Simeone