Model Details
- Developed by: Tuan Pham (FPTU HCM Student)
- Contact me at: [email protected] or [email protected]
- Looking for intern opportunity :D
- Model type: Llama2-7B Decoder-only
- Finetuned from model :
- meta-llama/Llama-2-7b
- bkai-foundation-models/vietnamese-llama2-7b-120GB
- yeen214/llama2_7b_merge_orcafamily.
- Bilingual support : English and Vietnamese
Model Description
This model is a proof of effort that one man can fine-tune his own model to reach SOTA.
Model Sources
- Repository:
- Paper: ...
- Demo: ...
- Here is a kaggle and colab script to quickly test the model:
Uses
Prompt template
[SYSTEM_PROMPT]
####### Instruction:
[INPUT]
%%%%%%% Response:
[RESPONSE]
Recommend keeping the system prompt in english.
How to Get Started with the Model
Use the code below to get started with the model.
import torch
from torch.cuda.amp import autocast
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer, pipeline
def prompt_format(system_prompt, instruction):
prompt = f"""{system_prompt}
####### Instruction:
{instruction}
%%%%%%% Response:
"""
return prompt
system_prompt = """
You're an AI Large Language Model developed(created) by an AI developer named Tuแบฅn, the architecture of you is decoder-based LM, your task are to think loudly step by step before give a good and relevant response
to the user request, answer in the language the user preferred.
The AI has been trained to answer questions, provide recommendations, and help with decision making. The AI thinks outside the box and follows the user requests
"""
instruction = "Xin chร o"
formatted_prompt = prompt_format(system_prompt, instruction)
print(formatted_prompt)
model_name = "1TuanPham/T-Llama"
model = AutoModelForCausalLM.from_pretrained(model_name,
torch_dtype=torch.bfloat16,
use_cache=True,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
streamer = TextStreamer(tokenizer, skip_special_tokens=True)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, streamer=streamer)
with autocast():
output_default = pipe(formatted_prompt, pad_token_id=50256, max_new_tokens=128)
Example outputs:
> Xin chร o
Xin chร o! Tรดi lร mแปt AI ฤฦฐแปฃc phรกt triแปn bแปi mแปt AI nhร phรกt triแปn tรชn lร Tuแบฅn. Tรดi ฤฦฐแปฃc thiแบฟt kแบฟ ฤแป giรบp ฤแปก ngฦฐแปi dรนng bแบฑng cรกch trแบฃ lแปi cรกc cรขu hแปi, ฤฦฐa ra ฤแป xuแบฅt vร hแป trแปฃ trong quรก trรฌnh ra quyแบฟt ฤแปnh.
Tรดi cรณ thแป hแป trแปฃ bแบกn bแบฑng cรกch nghฤฉ ra cรกc cรขu trแบฃ lแปi hay vร phรน hแปฃp cho cรกc cรขu hแปi cแปงa bแบกn.
> Viแปt Nam cรณ cร phรช gรฌ nแปi tiแบฟng thแบฟ giแปi ?
Cร phรช robusta cแปงa Viแปt Nam nแปi tiแบฟng trรชn toร n cแบงu. Loแบกi cร phรช nร y ฤฦฐแปฃc trแปng chแปง yแบฟu แป cรกc vรนng ฤแบฅt cao nhฦฐ Lรขm ฤแปng, Gia Lai, KonTum vร cรณ hฦฐฦกng vแป ฤแบญm ฤร , ฤแบฏng hฦกn so vแปi cร phรช Arabica.
Cร phรช robusta thฦฐแปng ฤฦฐแปฃc sแปญ dแปฅng ฤแป pha chแบฟ cรกc loแบกi cร phรช espresso vร phin truyแปn thแปng.
> Tรดi cรณ 500$ hรฃy build cho tรดi mแปt bแป PC
ฤแป xรขy dแปฑng mแปt bแป PC vแปi 500 USD, bแบกn cรณ thแป bแบฏt ฤแบงu bแบฑng cรกch chแปn cรกc thร nh phแบงn chรญnh nhฦฐ CPU, GPU, bแป nhแป vร bแป lฦฐu trแปฏ. Sau ฤรณ, bแบกn cรณ thแป thรชm cรกc thร nh phแบงn bแป sung nhฦฐ loa, bร n phรญm vร chuแปt ฤแป hoร n thiแปn hแป thแปng.
Dฦฐแปi ฤรขy lร mแปt sแป gแปฃi รฝ vแป cรกch bแบกn cรณ thแป cแบฅu thร nh mแปt bแป PC vแปi 500 USD:
1. CPU: Intel Core i5-12400 hoแบทc AMD Ryzen 5 5600X lร nhแปฏng lแปฑa chแปn tแปt ฤแป bแบฏt ฤแบงu.
2. GPU: Nvidia GeForce GTX 1650 hoแบทc AMD Radeon RX 570 lร nhแปฏng lแปฑa chแปn tแปt ฤแป bแบฏt ฤแบงu.
3. Bแป nhแป: 8GB RAM lร mแปt khแปi ฤแบงu tแปt.
4. Bแป lฦฐu trแปฏ: 500 GB แป cแปฉng SSD lร mแปt khแปi ฤแบงu tแปt.
5. Bร n phรญm vร chuแปt: Bร n phรญm vร chuแปt cรณ thแป ฤฦฐแปฃc mua riรชng hoแบทc cรนng mแปt bแป.
6. Loa: Mแปt loa cรณ thแป ฤฦฐแปฃc mua riรชng hoแบทc cรนng mแปt bแป vแปi bร n phรญm vร chuแปt.
> Cho tรดi 3 thแปฉc uแปng ฤแป giรบp ngแปง ngon hฦกn
Chแบฏc chแบฏn! Dฦฐแปi ฤรขy lร ba thแปฉc uแปng cรณ thแป giรบp bแบกn ngแปง ngon hฦกn:
1. Mแบญt ong vร chanh: Trแปn mแปt thรฌa mแบญt ong vแปi mแปt thรฌa nฦฐแปc cแปt chanh vร uแปng trฦฐแปc khi ฤi ngแปง. Mแบญt ong cรณ tรกc dแปฅng lร m dแปu, trong khi chanh cรณ chแปฉa chแบฅt chแปng oxy hรณa cรณ thแป giรบp cแบฃi thiแปn chแบฅt lฦฐแปฃng giแบฅc ngแปง.
2. Trร thแบฃo dฦฐแปฃc: Mแปt sแป loแบกi trร thแบฃo dฦฐแปฃc nhฦฐ hoa cรบc, hoa oแบฃi hฦฐฦกng vร hoa sen cรณ tรกc dแปฅng thฦฐ giรฃn vร giรบp bแบกn chรฌm vร o giแบฅc ngแปง. Hรฃy thแปญ pha mแปt แบฅm trร thแบฃo dฦฐแปฃc แบฅm vร uแปng trฦฐแปc khi ฤi ngแปง.
3. Sแปฏa แบฅm: Sแปฏa แบฅm cรณ chแปฉa tryptophan, mแปt loแบกi hormone giรบp bแบกn cแบฃm thแบฅy dแป
chแปu vร thฦฐ giรฃn. Nรณ cลฉng cรณ thแป giรบp thรบc ฤแบฉy sแบฃn xuแบฅt melatonin, hormone giรบp bแบกn chรฌm vร o giแบฅc ngแปง. Hรฃy thแปญ uแปng mแปt cแปc sแปฏa แบฅm trฦฐแปc khi ฤi ngแปง ฤแป giรบp bแบกn ngแปง ngon hฦกn.
The translated data has been thoroughly cleaned from the coding task, so in theory, the model should be able to code normally by incorporating the coding capabilities from the English coding examples task. (The translation will translate code definitions and variables.)
I would not recommend running any code generated from this model.
> Viแบฟt cho tรดi mแปt ฤoแบกn code bแบฑng python xรขy dแปฑng mแปt แปฉng dแปฅng bแบฑng Flask hiแปn thแป 'Xin chร o thแบฟ giแปi' trรชn mร n hรฌnh
ฤแป viแบฟt mแปt ฤoแบกn mรฃ Python xรขy dแปฑng mแปt แปฉng dแปฅng bแบฑng Flask hiแปn thแป 'Xin chร o thแบฟ giแปi' trรชn mร n hรฌnh, bแบกn cรณ thแป lร m theo cรกc bฦฐแปc sau:
1. Cร i ฤแบทt Flask vร cรกc thฦฐ viแปn cแบงn thiแบฟt:
```
pip install flask
```
2. Tแบกo mแปt แปฉng dแปฅng Flask mแปi:
```
app = Flask(__name__)
```
3. Thรชm mแปt route ฤแป hiแปn thแป 'Xin chร o thแบฟ giแปi':
```
@app.route('/')
def hello():
return 'Xin chร o thแบฟ giแปi!'
```
4. Khแปi tแบกo แปฉng dแปฅng Flask:
```
app.run()
```
5. Chแบกy แปฉng dแปฅng Flask:
```
app.run(debug=True)
```
6. Mแป trรฌnh duyแปt vร truy cแบญp vร o ฤแปa chแป IP hoแบทc tรชn mรกy chแปง cแปงa mรกy tรญnh hoแบทc mรกy chแปง nฦกi bแบกn ฤรฃ khแปi ฤแปng แปฉng dแปฅng Flask.
These examples are sub-optimal as they were generated with greedy decoding to ensure reproducibility. Here's the recommended configuration:
- top_k=50
- top_p=0.5
- temperature=0.5
- penalty_alpha=0.7
- repetition_penalty=1.15
Here is an example using the above generation config:
> Tรดi buแปn ngแปง quรก, cฦก mร thแปฉ 7 phแบฃi thi rแปi, nay lร rแบกng sรกng thแปฉ 6. Tรดi cรณ nรชn ฤi ngแปง khรดng trong khi tรดi mแปi xong chฦฐa ฤฦฐแปฃc 1/3 bร i ฤแป รดn ?
Nแบฟu bแบกn cแบฃm thแบฅy mแปt mแปi vร cแบงn nghแป ngฦกi thรฌ viแปc ฤi ngแปง sแปm sแบฝ rแบฅt quan trแปng ฤแปi vแปi sแปฉc khแปe cแปงa bแบกn. Tuy nhiรชn, nแบฟu bแบกn vแบซn muแปn hoร n thร nh mแปt phแบงn bร i tแบญp ฤแป chuแบฉn bแป cho kแปณ thi sแบฏp tแปi,
hรฃy cรขn nhแบฏc thแปi gian cรฒn lแบกi cแปงa mรฌnh vร ฦฐu tiรชn nhแปฏng nhiแปm vแปฅ quan trแปng nhแบฅt trฦฐแปc khi ฤi ngแปง. Nแบฟu bแบกn cแบฃm thแบฅy rแบฑng mรฌnh ฤรฃ lร m ฤแปง, hรฃy dร nh chรบt thแปi gian thฦฐ giรฃn hoแบทc thแปฑc hiแปn cรกc
hoแบกt ฤแปng khรกc ฤแป giรบp giแบฃm bแปt cฤng thแบณng vร tฤng cฦฐแปng sแปฑ tแปnh tรกo cแปงa bแบกn vร o buแปi sรกng hรดm sau. Cuแปi cรนng, quyแบฟt ฤแปnh cuแปi cรนng phแปฅ thuแปc vร o mแปฉc ฤแป nghiรชm trแปng cแปงa kแปณ thi vร khแบฃ nฤng quแบฃn
lรฝ thแปi gian cแปงa bแบกn.
# temperature=0.7
> Em ฤn cฦกm chฦฐa ?
Khรดng, tรดi khรดng ฤn cฦกm. Tรดi lร mแปt mรด hรฌnh ngรดn ngแปฏ mรกy ฤฦฐแปฃc phรกt triแปn bแปi mแปt nhร phรกt triแปn AI tรชn lร Tuแบฅn vร ฤฦฐแปฃc ฤร o tแบกo ฤแป trแบฃ lแปi cรกc cรขu hแปi vร cung cแบฅp phแบฃn hแปi hแปฏu รญch cho ngฦฐแปi dรนng.
Mแปฅc tiรชu cแปงa tรดi lร hแป trแปฃ ngฦฐแปi dรนng bแบฑng cรกch suy nghฤฉ trฦฐแปc khi ฤฦฐa ra nhแปฏng phแบฃn hแปi phรน hแปฃp vร chรญnh xรกc nhแบฅt cรณ thแป.
> Viแบฟt cho tรดi mแปt แปฉng dแปฅng flask in ra mร n hรฌnh 'Xin chร o Viแปt Nam!'
ฤแป viแบฟt แปฉng dแปฅng Flask hiแปn thแป "Xin chร o Viแปt Nam!" trรชn mร n hรฌnh, chรบng ta cรณ thแป lร m theo cรกc bฦฐแปc sau:
1. Cร i ฤแบทt Flask vร cรกc thฦฐ viแปn cแบงn thiแบฟt nhฦฐ BeautifulSoup4 vร jsonify. Bแบกn cรณ thแป sแปญ dแปฅng pip ฤแป cร i ฤแบทt chรบng:
```
pip install Flask
pip install beautifulsoup4
pip install jsonify
```
2. Tแบกo mแปt tแปp mแปi trong thฦฐ mแปฅc cแปงa bแบกn vแปi tรชn `app.py`. ฤรขy lร mรฃ mแบซu cฦก bแบฃn cแปงa แปฉng dแปฅng:
```python
from bs4 import BeautifulSoup
from jsonify import jsonify
import requests
app = Flask(__name__)
@app.route('/')
def home():
html_string = """<h1>Chร o mแปซng ฤแบฟn vแปi Viแปt Nam!</h1>
<p>ฤรขy lร mแปt trang web dร nh cho nhแปฏng ai muแปn tรฌm hiแปu vแป ฤแบฅt nฦฐแปc Viแปt Nam xinh ฤแบนp.</p>"""
soup = BeautifulSoup(html_string, 'html.parser')
print(soup)
return soup
if __name__ == '__main__':
app.run()
```
3. Khแปi tแบกo แปฉng dแปฅng bแบฑng cรกch chแบกy `app.py`:
```
python app.py
```
4. Mแป trรฌnh duyแปt web vร truy cแบญp vร o ฤแปa chแป IP hoแบทc tรชn mรกy chแปง cแปงa แปฉng dแปฅng (vรญ dแปฅ: http://localhost:5000). Trang ฤแบงu sแบฝ hiแปn thแป thรดng bรกo "Xin chร o Viแปt Nam!".
Lฦฐu รฝ rแบฑng ฤรขy chแป lร vรญ dแปฅ ฤฦกn giแบฃn vร bแบกn cรณ thแป tรนy chแปnh แปฉng dแปฅng ฤแป ฤรกp แปฉng nhu cแบงu cแปฅ thแป cแปงa mรฌnh.
> Liแปt kรช cho tรดi 5 thแปฉ cแบงn thiแบฟt ฤแป ฤem ฤi cแบฏm trแบกi
Chแบฏc chแบฏn! Dฦฐแปi ฤรขy lร nฤm vแบญt dแปฅng cแบงn thiแบฟt khi ฤi cแบฏm trแบกi:
1. Lแปu - Mแปt chiแบฟc lแปu chแบฏc chแบฏn vร thoแบฃi mรกi sแบฝ cung cแบฅp nฦกi trรบ แบฉn khแปi thแปi tiแบฟt vร cรดn trรนng.
2. Tรบi ngแปง - Mแปt tรบi ngแปง แบฅm รกp vร thoแบฃi mรกi sแบฝ giรบp bแบกn cรณ giแบฅc ngแปง ngon trong khi แป ngoร i trแปi.
3. Bแบฟp di ฤแปng - Mแปt bแบฟp di ฤแปng nhฦฐ bรฌnh ฤun hoแบทc bแบฟp cแปn sแบฝ giรบp bแบกn nแบฅu ฤn vร giแปฏ แบฅm thแปฉc ฤn cแปงa mรฌnh.
4. ฤรจn pin - Mแปt ฤรจn pin mแบกnh mแบฝ sแบฝ giรบp chiแบฟu sรกng khu vแปฑc cแบฏm trแบกi cแปงa bแบกn vร o ban ฤรชm.
5. Bแบฃn ฤแป vร la bร n - Bแบฃn ฤแป vร la bร n sแบฝ giรบp bแบกn ฤแปnh hฦฐแปng chรญnh xรกc ฤแบฟn ฤแปa ฤiแปm cแบฏm trแบกi cแปงa bแบกn vร quay lแบกi nhร nแบฟu cแบงn.
> Viแบฟt cho tรดi mแปt bร i thฦก vแป thร nh phแป ฤร Lแบกt
Thร nh phแป sฦฐฦกng mรน, nฦกi thiรชn nhiรชn thanh bรฌnh,
Nฦกi nhแปฏng bรดng hoa khoe sแบฏc vร cรขy xanh tฦฐฦกi tแปt,
Mแปt แปc ฤแบฃo giแปฏa lรฒng ฤแบฅt, mแปt xแปฉ sแป thแบงn tiรชn,
Nฦกi mแปi lo lแบฏng ฤแปu tan biแบฟn trong khรดng khรญ.
Nhแปฏng con ฤฦฐแปng nhแป ฤan xen qua nhแปฏng ngแปn ฤแปi thoai thoแบฃi,
Vร nhแปฏng ngรดi nhร cแป kรญnh, vแปi nhแปฏng bแปฉc tฦฐแปng mร u nรขu แบฅm รกp,
Nฦกi nhแปฏng ngฦฐแปi dรขn thรขn thiแปn chร o ฤรณn bแบกn bแบฑng nแปฅ cฦฐแปi,
Vร tiแบฟng chim hรณt lรญu lo nhฦฐ lแปi chร o cแปงa thiรชn nhiรชn.
Nhแปฏng dรฒng suแปi chแบฃy รชm ฤแปm, nhแปฏng thรกc nฦฐแปc ฤแป xuแปng,
Vร nhแปฏng hแป nฦฐแปc lแบฅp lรกnh, nhแปฏng cแบฃnh tฦฐแปฃng ngoแบกn mแปฅc,
Nฦกi sแปฑ tฤฉnh lแบทng ngแปฑ trแป, nฦกi bรฌnh yรชn ngแปฑ trแป,
Nฦกi tรขm hแปn ฤฦฐแปฃc chแปฏa lร nh vร cแบฃm xรบc ฤฦฐแปฃc giแบฃi phรณng.
แป ฤรขy, thแปi gian dฦฐแปng nhฦฐ ฤแปฉng yรชn,
Nฦกi nhแปฏng kแปท niแปm vแบซn cรฒn nguyรชn vแบนn,
Nฦกi nhแปฏng mแปi quan hแป bแปn chแบทt ฤฦฐแปฃc nuรดi dฦฐแปกng,
Nฦกi tรฌnh yรชu vร hรฒa bรฌnh chiแบฟm vแป trรญ hร ng ฤแบงu.
Thแบฟ giแปi bรชn ngoร i cรณ vแบป xa xรดi,
Nhฦฐng แป ฤรขy, chแป cแบงn bฦฐแปc chรขn ฤi,
Nฦกi bแบงu trแปi rแปng lแปn, nhแปฏng ฤรกm mรขy cao chรณt vรณt,
Vร nhแปฏng tia nแบฏng vร ng rแปฑc rแปก, sฦฐแปi แบฅm trรกi tim chรบng ta.
รi, thร nh phแป sฦฐฦกng mรน, nฦกi thiรชn nhiรชn thanh bรฌnh,
Nฦกi nhแปฏng giแบฅc mฦก trแป thร nh hiแปn thแปฑc, nฦกi trรกi tim trร n ngแบญp niแปm vui,
Nฦกi tรฌnh yรชu vร hรฒa bรฌnh ngแปฑ trแป, nฦกi tรฌnh yรชu vร hรฒa bรฌnh ngแปฑ trแป.
Note: 120GB of pre-trained Vietnamese data might not be enough for a general question about Vietnamese events.
Training Details
Hardware Type:
- GPU: VGA NVIDIA Tesla P100 16GB
- SYSTEM RAM: 32GB
Hours used: ~47.5 days Approx*
Training Data
- BactrianX
- OpenOrca_translated
- WizardLM_70k_translated
- TigerLabMathInstruct_translated_vi
- GradeSchoolMathInstruct_translated
- vilm_lima-vi
- MTEngVietnamese
- databricks_dolly15k_translated
- AlpacaCleaned_translated
- databricks_dolly15k
- OpenOrca
- GradeSchoolMathInstruct
- AlpacaCleaned
- WebglmQA
Training Procedure
- Learning rate: 2e-5 cosine
- Optimizer: PagedLion8bit
- QLora: rank: 64 /Q: 4-bit
- Batch size: 2
- Gradient accumulation: 128
- Effective batch size: 256
- 250k examples of 70% Vietnamese 30% English for 3.37 epoch
- 350k examples of 60% Vietnamese 40% English for 1.4 epoch
Training loss
Last 10 runs:
Each line is 12 hours
All run plots:
A bad way to visualize i know...
Evaluation
Our model currently sits at TOP-5 on the VMLU benchmark
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 54.34 |
| AI2 Reasoning Challenge (25-Shot) | 54.18 |
| HellaSwag (10-Shot) | 76.48 |
| MMLU (5-Shot) | 47.98 |
| TruthfulQA (0-shot) | 46.47 |
| Winogrande (5-shot) | 71.27 |
| GSM8k (5-shot) | 29.64 |
Citation
@online{t-llama,
author = {Pham Minh Tuan},
title = {T-Llama: A New Language Model for Vietnamese}, % Placeholder
year = 2024,
url = {https://github.com/vTuanpham/Vietnamese_QA_System}
}
- Downloads last month
- 7
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
๐
Ask for provider support
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard54.180
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard76.480
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard47.980
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard46.470
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard71.270
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard29.640