hindi-tokenizer / README.md
atharvanighot's picture
Update README.md
8721db5 verified
metadata
library_name: transformers
tags: []

Model Card for Model ID

Llama 2 Tokenizer with Additional 25104 Hindi Tokens which makes it much better on Hindi Data