Edit model card

Model Card for Model ID

The model is obtained by performing language adaptation on the original bloom-1b7 model. In detail, we continued the pre-training on Italian-specific data without adaptation of the vocabulary. We use about 2.8M documents obtained from Italian Wikimedia dumps (swap-uniba/itwiki-march-2024). The model is trained for one epoch using LoRA and SFT.

Model Details

Model Description

  • Developed by: SWAP Research Group, Department of Computer Science, University of Bari Aldo Moro
  • Model type: BLOOM
  • Language(s) (NLP): Italian
  • License: bigscience-bloom-rail-1.0
  • Finetuned from model [optional]: bloom-1b7

Training Details

Training Data

2.8M documents obtained from Italian Wikimedia dumps (swap-uniba/itwiki-march-2024).

Training Procedure

LoRA and SFT.

Training Hyperparameters

  • Training regime: fp16

Citation [optional]

BibTeX:

APA:

Model Card Authors [optional]

Pierpaolo Basile, University of Bari Aldo Moro, Italy.

Model Card Contact

Pierpaolo Basile, University of Bari Aldo Moro, Italy.

Downloads last month
12
Safetensors
Model size
1.72B params
Tensor type
FP16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for swap-uniba/bloom-1b7-it

Adapters
5 models

Dataset used to train swap-uniba/bloom-1b7-it

Collection including swap-uniba/bloom-1b7-it