DictaBERT-char: A Character-Level BERT-Base model for Hebrew.

DictaBERT-char is a BERT-style language model for Hebrew, based on the BERT-base architecture with a character level tokenizer. The model based on the BERT-Large architecture is available here.

This model is released to the public in this 2025 W-NUT paper: Avi Shmidman and Shaltiel Shmidman, "Restoring Missing Spaces in Scraped Hebrew Social Media", The 10th Workshop on Noisy and User-generated Text (W-NUT), 2025

This is the base model pretrained with the masked-language-modeling objective.

Sample usage:

from transformers import AutoModelForMaskedLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('dicta-il/dictabert-char')
model = AutoModelForMaskedLM.from_pretrained('dicta-il/dictabert-char')

model.eval()

sentence = 'בשנת 1948 השלים אפרים קישון את מחקרו בפיסול מתכת וב[MASK]ולדות האמנות והחל לפרסם מאמרים הומוריסטיים'

output = model(tokenizer.encode(sentence, return_tensors='pt'))
# the [MASK] is the 52nd token (including [CLS])
import torch
top_arg = torch.argmax(output.logits[0, 52, :])
print(tokenizer.convert_ids_to_tokens([top_arg])) # should print ['ת'] 

Citation

If you use DictaBERT-char in your research, please cite Restoring Missing Spaces in Scraped Hebrew Social Media

BibTeX:

@inproceedings{shmidman2025restoring,
  author    = {Avi Shmidman and Shaltiel Shmidman},
  title     = {Restoring Missing Spaces in Scraped Hebrew Social Media},
  booktitle = {The 10th Workshop on Noisy and User-generated Text (W-NUT)},
  year      = {2025}
}

License

Shield:

This work is licensed under a Creative Commons Attribution 4.0 International License.