Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,9 @@ base_model:
|
|
9 |
---
|
10 |
|
11 |
# Motivation :
|
12 |
-
|
|
|
|
|
13 |
|
14 |
# Training :
|
15 |
This model is the 2B version. It was trained for 2 days on 1 A100 GPU using LoRA with a rank of 128, a learning rate of 1e-4, and a cosine learning rate schedule.
|
@@ -33,3 +35,5 @@ This model is the 2B version. It was trained for 2 days on 1 A100 GPU using LoRA
|
|
33 |
| RACE | 36.19 |
|
34 |
| SCIQ | 55.78 |
|
35 |
| TOXIGEN | 78.29 |
|
|
|
|
|
|
9 |
---
|
10 |
|
11 |
# Motivation :
|
12 |
+
|
13 |
+
The goal of the project was to adapt large language models for the Arabic language and create a new state-of-the-art Arabic LLM. Due to the scarcity of Arabic instruction fine-tuning data, not many LLMs have been trained specifically in Arabic, which is surprising given the large number of Arabic speakers.
|
14 |
+
Our final model was trained on a high-quality instruction fine-tuning (IFT) dataset, generated synthetically and then evaluated using the Hugging Face Arabic leaderboard.
|
15 |
|
16 |
# Training :
|
17 |
This model is the 2B version. It was trained for 2 days on 1 A100 GPU using LoRA with a rank of 128, a learning rate of 1e-4, and a cosine learning rate schedule.
|
|
|
35 |
| RACE | 36.19 |
|
36 |
| SCIQ | 55.78 |
|
37 |
| TOXIGEN | 78.29 |
|
38 |
+
|
39 |
+
Please refer to https://github.com/Slim205/Arabicllm/ for more details.
|