|
--- |
|
language: |
|
- en |
|
- hi |
|
tags: |
|
- Llama3 |
|
- Pytorch |
|
- Bilingual |
|
pipeline_tag: text-generation |
|
--- |
|
# SandLogic Technologies - Quantized LLama3-Gaja-Hindi-8B Models |
|
|
|
## Model Description |
|
|
|
We have quantized the LLama3-Gaja-Hindi-8B model into two variants: |
|
|
|
1. Q5_KM |
|
2. Q4_KM |
|
|
|
These quantized models offer improved efficiency while maintaining performance. |
|
|
|
Discover our full range of quantized language models by visiting our [SandLogic Lexicon](https://github.com/sandlogic/SandLogic-Lexicon) GitHub. To learn more about our company and services, check out our website at [SandLogic](https://www.sandlogic.com). |
|
|
|
## Original Model Information |
|
|
|
- **Name**: [LLama3-Gaja-Hindi-8B-v0.1](https://huggingface.co/Cognitive-Lab/LLama3-Gaja-Hindi-8B-v0.1) |
|
- **Developer**: Cognitivelab.in |
|
- **Base Model**: Llama3-8B |
|
- **Model Type**: Bilingual English/Hindi language model |
|
- **Parameters**: 8 billion |
|
- **Specialization**: Natural language understanding, particularly for instructional pairs |
|
|
|
## Model Capabilities |
|
|
|
LLama3-Gaja-Hindi-8B is designed for bilingual (English/Hindi) natural language understanding tasks, with a focus on: |
|
|
|
- Responding appropriately in either English or Hindi based on user prompts |
|
- Understanding and generating instructional content in both languages |
|
- Handling a variety of natural language processing tasks across both languages |
|
|
|
## Training Approach |
|
|
|
The model underwent supervised fine-tuning with low-rank adaptation, focusing on bilingual instruct fine-tuning. The training data consisted of a curated dataset of translated instructional pairs. |
|
|
|
## Use Cases |
|
|
|
1. **Bilingual Conversational AI**: Chatbots and virtual assistants with English/Hindi capabilities |
|
2. **Language Learning Tools**: Interactive platforms for English and Hindi learners |
|
3. **Content Translation**: Assistance in translating between English and Hindi, especially for instructional materials |
|
4. **Cross-lingual Information Retrieval**: Enabling queries in one language with responses in either English or Hindi |
|
5. **Cultural Context Understanding**: Helping users grasp cultural nuances in both languages |
|
6. **Multilingual Customer Support**: Powering customer service applications for diverse user bases |
|
|
|
## Model Variants |
|
|
|
We offer two quantized versions of the LLama3-Gaja-Hindi-8B model: |
|
|
|
1. **Q5_KM**: 5-bit quantization using the KM method |
|
2. **Q4_KM**: 4-bit quantization using the KM method |
|
|
|
These quantized models aim to reduce model size and improve inference speed while maintaining performance as close to the original model as possible. |
|
|
|
## Input and Output |
|
|
|
- **Input**: Text prompts or instructions in either English or Hindi |
|
- **Output**: Generated text responses in the same language as the input, or as specified in the prompt |
|
|
|
## Usage |
|
|
|
```bash |
|
pip install llama-cpp-python |
|
``` |
|
Please refer to the llama-cpp-python [documentation](https://llama-cpp-python.readthedocs.io/en/latest/) to install with GPU support. |
|
|
|
### Basic Text Completion |
|
Here's an example demonstrating how to use the high-level API for basic text completion: |
|
|
|
```bash |
|
from llama_cpp import Llama |
|
|
|
llm = Llama( |
|
model_path="./models/7B/LLama3-Gaja-Hindi-8B-v0.1.gguf", |
|
verbose=False, |
|
# n_gpu_layers=-1, # Uncomment to use GPU acceleration |
|
# n_ctx=2048, # Uncomment to increase the context window |
|
) |
|
|
|
output = llm.create_chat_completion( |
|
messages =[ |
|
{ |
|
"role": "system", |
|
"content": """ You are an AI assistant trained on top of Llama 3 Large language model (LLM), proficient in English and Hindi. You can respond in both languages based on the user's request.""" |
|
|
|
, |
|
}, |
|
{"role": "user", "content": "Write an poem in hindi"}, |
|
] |
|
) |
|
|
|
print(output["choices"][0]['message']['content']) |
|
``` |
|
|
|
## Download |
|
You can download `Llama` models in `gguf` format directly from Hugging Face using the `from_pretrained` method. This feature requires the `huggingface-hub` package. |
|
|
|
To install it, run: `pip install huggingface-hub` |
|
|
|
```bash |
|
from llama_cpp import Llama |
|
|
|
llm = Llama.from_pretrained( |
|
repo_id="SandLogicTechnologies/LLama3-Gaja-Hindi-8B-GGUF", |
|
filename="*llama3-gaja-hindi-8b-v0.1.Q5_K_M.gguf", |
|
verbose=False |
|
) |
|
``` |
|
By default, from_pretrained will download the model to the Hugging Face cache directory. You can manage installed model files using the huggingface-cli tool. |
|
|
|
|
|
## Ethical Considerations |
|
|
|
Users should be aware of potential biases in the model's outputs, especially when dealing with cultural or linguistic nuances. Always review and validate the model's responses for sensitive applications. |
|
|
|
|
|
|
|
## Acknowledgements |
|
|
|
We thank Cognitivelab.in for developing the original LLama3-Gaja-Hindi-8B model and the creators of Llama3 for their foundational work.Special thanks to Georgi Gerganov and the entire llama.cpp development team for their outstanding contributions. |
|
|
|
## Contact |
|
|
|
For any inquiries or support, please contact us at [email protected] or visit our [support page](https://www.sandlogic.com/LingoForge/support). |
