luigisaetta/squad_it_xxl_cased_hub1

squad_it_xxl_cased

This is a model, based on BERT trained on cased Italian, that can be used for Extractive Q&A on Italian texts.

Model description

This model has been trained on squad_it dataset starting from the pre-trained model dbmdz/bert-base-italian-xxl-cased.

These are the metrics computed on evaluation set:

• EM: 63.95
• F1: 75.27

How to use

from transformers import pipeline

pipe_qa = pipeline('question-answering', model='luigisaetta/squad_it_xxl_cased_hub1')

pipe_qa(context="Io sono nato a Napoli. Il mare bagna Napoli. Napoli è la più bella città del mondo", 
        question="Qual è la più bella città del mondo?")

Intended uses & limitations

This model can be used for Extractive Q&A on Italian Text

Training and evaluation data

squad_it

Training procedure

see code in this NoteBook

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 1234
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2
• mixed_precision_training: Native AMP

Framework versions

• Transformers 4.20.0.dev0
• Pytorch 1.9.0
• Datasets 1.11.0
• Tokenizers 0.12.1