hindi-tokenizer / README.md
atharvanighot's picture
Update README.md
8721db5 verified
---
library_name: transformers
tags: []
---
# Model Card for Model ID
<!-- Provide a quick summary of what the model is/does. -->
Llama 2 Tokenizer with Additional 25104 Hindi Tokens which makes it much better on Hindi Data