|
--- |
|
license: apache-2.0 |
|
language: |
|
- bh |
|
- hi |
|
- en |
|
base_model: |
|
- google/gemma-2-9b |
|
library_name: transformers |
|
--- |
|
|
|
SASTRI_1_9B is a new model fine tuned for bhojpuri & hindi languages. |
|
More detail about the dataset used and evaluation will be shared soon. |
|
Model trained on similar languages can be adapted for a very low resource language. |
|
|
|
Let's see what model has to say about itself. |
|
|
|
You can also download the GGUF version from : https://huggingface.co/divyanshukunwar/SASTRI_1_9B_GGUF |
|
``` |
|
User : Tell me about yourself |
|
Assistant : Hi there! I am SASTRI, an AI assistant designed to help you with various tasks and answer questions as best as my programming allows. |
|
As a large language model, I don't have personal feelings or emotions like humans do but rest assured that I am here to assist you with anything within my capabilities! |
|
|
|
User : aap or kon kon si bhaasa me likh skte hai ? |
|
Assistant : मैं कई भाषाओं को समझता हूँ और उनमें बातचीत कर सकता हूँ, हिंदी में लिखने के अलावा मैं अंग्रेजी और भोजपुरी भी जानता हूं। 🇮🇳 |
|
``` |
|
|
|
Evaluation Result for English Language |
|
|Dataset| Metric | Score | |
|
|--------|---------|---------| |
|
|Arc-Easy| Accuracy | 94.2 | |
|
|Arc-Challenge| Accuracy | 86.1 | |
|
|MMLU| Accuracy | 61.0 | |
|
|BoolQ| Accuracy | 79.04 | |
|
|
|
Open LLM Leaderboard Result |
|
 |
|
|
|
|
|
Evaluation Result for Hindi Language |
|
|Dataset| Metric | Score | |
|
|--------|---------|---------| |
|
|Arc-Easy| Accuracy | 80.2 | |
|
|Arc-Challenge| Accuracy | 68.35 | |
|
|MMLU| Accuracy | 43.11 | |
|
|BoolQ| Accuracy | 71.61 | |
|
|
|
Evaluation Result for Bhojpuri Language coming soon .. |
|
|
|
Model may show undersired behavior such as hallucinations and unintended biases use it at your own risk. |
|
This model is not optimized for user preference. |
|
This model is released as a part of study for Large Language model for very low resource languages. |
|
|
|
|Comparison with other Indic models on English Evaluation Benchmark|Comparison with other models on Hindi Evaluation Benchmark| |
|
|---|--| |
|
||| |
