|
--- |
|
license: creativeml-openrail-m |
|
language: |
|
- en |
|
- de |
|
- fr |
|
- it |
|
- pt |
|
- hi |
|
- es |
|
- th |
|
base_model: |
|
- prithivMLmods/Triangulum-10B |
|
pipeline_tag: text-generation |
|
library_name: transformers |
|
tags: |
|
- triangulum_10b |
|
- sft |
|
- chain_of_thought |
|
- ollama |
|
- text-generation-inference |
|
- llama_for_causal_lm |
|
--- |
|
 |
|
|
|
<pre align="center"> |
|
__ .__ .__ |
|
_/ |_ _______ |__|_____ ____ ____ __ __ | | __ __ _____ |
|
\ __\\_ __ \| |\__ \ / \ / ___\ | | \| | | | \ / \ |
|
| | | | \/| | / __ \_| | \/ /_/ >| | /| |__| | /| Y Y \ |
|
|__| |__| |__|(____ /|___| /\___ / |____/ |____/|____/ |__|_| / |
|
\/ \//_____/ \/ |
|
</pre> |
|
|
|
# **Triangulum 10B: Multilingual Large Language Models (LLMs)** |
|
|
|
Triangulum 10B is a collection of pretrained and instruction-tuned generative models, designed for multilingual applications. These models are trained using synthetic datasets based on long chains of thought, enabling them to perform complex reasoning tasks effectively. |
|
|
|
# **Key Features** |
|
|
|
- **Foundation Model**: Built upon LLaMA's autoregressive language model, leveraging an optimized transformer architecture for enhanced performance. |
|
|
|
- **Instruction Tuning**: Includes supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align model outputs with human preferences for helpfulness and safety. |
|
|
|
- **Multilingual Support**: Designed to handle multiple languages, ensuring broad applicability across diverse linguistic contexts. |
|
|
|
# **Training Approach** |
|
|
|
1. **Synthetic Datasets**: Utilizes long chain-of-thought synthetic data to enhance reasoning capabilities. |
|
2. **Supervised Fine-Tuning (SFT)**: Aligns the model to specific tasks through curated datasets. |
|
3. **Reinforcement Learning with Human Feedback (RLHF)**: Ensures the model adheres to human values and safety guidelines through iterative training processes. |
|
|
|
# **How to use with transformers** |
|
|
|
Starting with `transformers >= 4.43.0` onward, you can run conversational inference using the Transformers `pipeline` abstraction or by leveraging the Auto classes with the `generate()` function. |
|
|
|
Make sure to update your transformers installation via `pip install --upgrade transformers`. |
|
|
|
```python |
|
import torch |
|
from transformers import pipeline |
|
|
|
model_id = "prithivMLmods/Triangulum-10B" |
|
pipe = pipeline( |
|
"text-generation", |
|
model=model_id, |
|
torch_dtype=torch.bfloat16, |
|
device_map="auto", |
|
) |
|
messages = [ |
|
{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"}, |
|
{"role": "user", "content": "Who are you?"}, |
|
] |
|
outputs = pipe( |
|
messages, |
|
max_new_tokens=256, |
|
) |
|
print(outputs[0]["generated_text"][-1]) |
|
``` |
|
# **Use Cases** |
|
|
|
- Multilingual content generation |
|
- Question answering and dialogue systems |
|
- Text summarization and analysis |
|
- Translation and localization tasks |
|
|
|
# **Technical Details** |
|
|
|
Triangulum 10B employs a state-of-the-art autoregressive architecture inspired by LLaMA. The optimized transformer framework ensures both efficiency and scalability, making it suitable for a variety of use cases. |
|
|
