Labour-Law-SA-QA

This model is a fine-tuned version of aubmindlab/bert-base-arabert on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1740

Model description

The Labour-Law-SA-QA model is a fine-tuned version of the aubmindlab/bert-base-arabert model on a custom dataset of questions and answers about labour law in Saudi Arabia. The model is trained to predict the answer to a question given the question text and the context of the surrounding text.

Intended uses & limitations

The Labour-Law-SA-QA model is intended to be used to answer questions about labour law in Saudi Arabia. The model is not intended to be used for legal advice, and it should not be used to replace the advice of a qualified lawyer. The model is limited by the quality of the training data. If the training data is not representative of the real-world questions that the model will be asked, then the model's performance will be degraded.

Training and evaluation data

The Labour-Law-SA-QA model was trained on a custom dataset of questions and answers about labour law in Saudi Arabia. The dataset was created by collecting questions from a variety of sources, including government websites. The dataset was then manually cleaned and verified to ensure that the questions and answers were accurate and relevant.

Training procedure

The Labour-Law-SA-QA model was trained using the Hugging Face Transformers library: https://huggingface.co/transformers/. The model was fine-tuned using the Adam optimizer with a learning rate of 2e-05. The model was trained for 9 epochs, and the training was stopped early when the validation loss did not improve for 3 consecutive epochs.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 9

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	34	1.6275
No log	2.0	68	1.4822
No log	3.0	102	1.4659
No log	4.0	136	1.3038
No log	5.0	170	1.3173
No log	6.0	204	1.1665
No log	7.0	238	1.1344
No log	8.0	272	1.1346
No log	9.0	306	1.1740

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1
• Datasets 2.14.4
• Tokenizers 0.13.3