metadata
title: README
emoji: 📊
colorFrom: purple
colorTo: gray
sdk: static
pinned: false
Multilingual language models are typically large, requiring significant computational resources.
Can we create multilingual models that maintain performance comparable to their larger models while reducing size, latency and inference speeds?
Techniques:
- Pruning
- SparseGPT | GitHub
- ShortGPT | KLDBasedPruning & Perplexity Sensivities
- Knowledge Distillation
- DistillKit | GitHub
- Distil-Whisper based method
- On policy distillation of language models
- Minitron: Compact Language models via Pruning & Knowledge Distillation
- DistiLLM: Towards Streamlined Distillation for Large Language Models
- Quantization
- KV Cache Compression
- Fine-Tuning | GitHub
Datasets:
Initial 7 datasets unified, having 6.62M rows which includes the following:
- Bangla_Alpaca_Orca : Bangle
- Urdu_Instruct_News_Article_Generation: Urdu
- Urdu_Instruct_News_Headline_Generation: Urdu
- Urdu_Instruct_News_Category_Classification: Urdu
- cidar: Arabic
- Six_Millions_Instruction_Dataset_For_Arabic_Llm_Ft: Arabic
- instructv3: English
Get in touch with the team:
- Mayank Bhaskar -> [email protected]
- Ahmad Anis -> [email protected]
- Drishti Sharma -> [email protected]
- Vishnu Vardhan -> [email protected]
- Yaya -> [email protected]
- Shayekh Bin Islam -> [email protected]