Italian ModernBERT
Model Description
Italian ModernBERT (DeepMount00/Italian-ModernBERT-base) is a specialized Italian language version of ModernBERT, specifically pre-trained on Italian text corpora. This model is designed exclusively for Italian language tasks.
Key Features
- Architecture: Based on ModernBERT-base (22 layers, 149M parameters)
- Context Length: 8,192 tokens
- Language: Italian-only
- Tokenizer: Custom tokenizer optimized for Italian language
- Training: Pre-trained on Italian text corpus
Technical Details
- Uses Rotary Positional Embeddings (RoPE)
- Implements Local-Global Alternating Attention
- Supports Flash Attention 2 for optimal performance
- No token type IDs required
Limitations
- Optimized only for Italian language processing
- Not suitable for other languages
- May reflect biases present in training data
- Downloads last month
- 422
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.