Slim205 commited on
Commit
15cdc8a
1 Parent(s): a42e5ad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -9,7 +9,9 @@ base_model:
9
  ---
10
 
11
  # Motivation :
12
- The goal of this project is to adapt large language models for the Arabic language. Due to the scarcity of Arabic instruction fine-tuning data, the focus is on creating a high-quality instruction fine-tuning (IFT) dataset. The project aims to finetune models on this dataset and evaluate their performance across various benchmarks.
 
 
13
 
14
  # Training :
15
  This model is the 2B version. It was trained for 2 days on 1 A100 GPU using LoRA with a rank of 128, a learning rate of 1e-4, and a cosine learning rate schedule.
@@ -33,3 +35,5 @@ This model is the 2B version. It was trained for 2 days on 1 A100 GPU using LoRA
33
  | RACE | 36.19 |
34
  | SCIQ | 55.78 |
35
  | TOXIGEN | 78.29 |
 
 
 
9
  ---
10
 
11
  # Motivation :
12
+
13
+ The goal of the project was to adapt large language models for the Arabic language and create a new state-of-the-art Arabic LLM. Due to the scarcity of Arabic instruction fine-tuning data, not many LLMs have been trained specifically in Arabic, which is surprising given the large number of Arabic speakers.
14
+ Our final model was trained on a high-quality instruction fine-tuning (IFT) dataset, generated synthetically and then evaluated using the Hugging Face Arabic leaderboard.
15
 
16
  # Training :
17
  This model is the 2B version. It was trained for 2 days on 1 A100 GPU using LoRA with a rank of 128, a learning rate of 1e-4, and a cosine learning rate schedule.
 
35
  | RACE | 36.19 |
36
  | SCIQ | 55.78 |
37
  | TOXIGEN | 78.29 |
38
+
39
+ Please refer to https://github.com/Slim205/Arabicllm/ for more details.