🇮🇳 Hindi Text Generative Model (HTGM)
A GPT-style Hindi Language Model trained from scratch.
🚀 Model Overview
Hindi Text Generative Model (HTGM) is a Transformer-based GPT architecture model trained from scratch on large-scale Hindi text data.
This model is designed to generate Hindi text, complete sentences, answer prompts, and assist in Hindi NLP tasks.
The model is fully public and anyone can use it freely under the Apache 2.0 License.
👨💻 Creator
Name: Mahesh Editor
Age: 17
Developed by: Mahesh Editor (with AI assistance)
Email: [email protected]
This model was built end-to-end by Mahesh Editor, including:
- Data collection
- Data cleaning
- Tokenizer training
- Model architecture design
- Training pipeline creation
- Checkpoint management
- Evaluation system
AI was used as a research and development assistant during the process.
🧠 Architecture
- Model Type: GPT-style Transformer
- Layers: 12
- Hidden Size: 768
- Attention Heads: 12
- Feedforward Size: 3072
- Context Length: 768 tokens
- Vocabulary Size: 64,000 (SentencePiece BPE)
Framework: PyTorch + HuggingFace Transformers
📚 Training Data
- Source: AI4Bharat Hindi datasets
- Raw Size: ~9GB
- Cleaned Size: ~8.98GB
- Training Duration (this release): ~50 GPU hours
- Training Type: From scratch (not fine-tuned)
Data was cleaned and preprocessed before training.
⚙️ Training Details
- Optimizer: AdamW
- Scheduler: Cosine with warmup
- Weight Decay: 0.1
- Gradient Accumulation used
- Mixed Precision (AMP) training
- Multi-checkpoint system
🎯 Intended Use
This model can be used for:
- Hindi text generation
- Prompt completion
- Educational experiments
- Research projects
- Hindi NLP development
- Building chat systems
- Fine-tuning for downstream tasks
⚠️ Limitations
- The model may produce repetitive or incoherent outputs.
- It is not instruction-tuned.
- It may require further fine-tuning (SFT / RAG) for high-quality chat performance.
📜 License
Apache License 2.0
You are free to:
- Use
- Modify
- Distribute
- Build commercial applications
As long as the Apache 2.0 license terms are followed.
📌 Tags
hindi, gpt, transformer, llm, text-generation, nlp, ai, pytorch, research, language-model
🔥 Vision
HTGM is an independent Hindi LLM project built from scratch with the goal of contributing to open Hindi AI development.
Future versions may include:
- Larger training runs
- Better structured datasets
- Instruction tuning
- Retrieval Augmented Generation (RAG)
- Improved alignment
🤝 Contact
For collaboration, research, feedback or contribution:
Built with passion by Mahesh Editor.