arabert_cross_development_task1_fold6

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6885
• Qwk: 0.4652
• Mse: 0.6865

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1333	2	2.5610	0.0172	2.5631
No log	0.2667	4	1.3092	0.0773	1.3077
No log	0.4	6	0.6287	0.4047	0.6286
No log	0.5333	8	0.7953	0.4436	0.7955
No log	0.6667	10	0.6050	0.3695	0.6046
No log	0.8	12	0.5414	0.3312	0.5412
No log	0.9333	14	0.3774	0.5165	0.3781
No log	1.0667	16	0.3713	0.5340	0.3720
No log	1.2	18	0.3244	0.5756	0.3250
No log	1.3333	20	0.3167	0.6313	0.3171
No log	1.4667	22	0.3711	0.6258	0.3708
No log	1.6	24	0.3620	0.6705	0.3621
No log	1.7333	26	0.3546	0.6894	0.3547
No log	1.8667	28	0.3904	0.5542	0.3900
No log	2.0	30	0.4427	0.4960	0.4420
No log	2.1333	32	0.4085	0.5287	0.4081
No log	2.2667	34	0.3250	0.6490	0.3251
No log	2.4	36	0.3625	0.7352	0.3630
No log	2.5333	38	0.3719	0.7111	0.3719
No log	2.6667	40	0.4472	0.5221	0.4462
No log	2.8	42	0.5148	0.4902	0.5136
No log	2.9333	44	0.4050	0.5405	0.4043
No log	3.0667	46	0.3378	0.6397	0.3376
No log	3.2	48	0.3496	0.5949	0.3493
No log	3.3333	50	0.3447	0.6021	0.3444
No log	3.4667	52	0.3460	0.5885	0.3457
No log	3.6	54	0.3308	0.6420	0.3306
No log	3.7333	56	0.3377	0.6102	0.3374
No log	3.8667	58	0.3579	0.5907	0.3575
No log	4.0	60	0.3816	0.5790	0.3810
No log	4.1333	62	0.4251	0.5479	0.4242
No log	4.2667	64	0.4317	0.5444	0.4308
No log	4.4	66	0.5698	0.4745	0.5684
No log	4.5333	68	0.5338	0.4833	0.5325
No log	4.6667	70	0.4617	0.5347	0.4607
No log	4.8	72	0.4937	0.4804	0.4925
No log	4.9333	74	0.5167	0.4702	0.5156
No log	5.0667	76	0.4987	0.4743	0.4977
No log	5.2	78	0.5594	0.4792	0.5582
No log	5.3333	80	0.5679	0.4788	0.5667
No log	5.4667	82	0.5202	0.4874	0.5190
No log	5.6	84	0.5297	0.4891	0.5284
No log	5.7333	86	0.4835	0.4882	0.4825
No log	5.8667	88	0.5151	0.4891	0.5140
No log	6.0	90	0.6542	0.4666	0.6526
No log	6.1333	92	0.7260	0.4338	0.7242
No log	6.2667	94	0.5806	0.4788	0.5791
No log	6.4	96	0.4674	0.5006	0.4664
No log	6.5333	98	0.4558	0.5000	0.4549
No log	6.6667	100	0.5518	0.4839	0.5504
No log	6.8	102	0.6844	0.4344	0.6825
No log	6.9333	104	0.6391	0.4542	0.6374
No log	7.0667	106	0.5221	0.5072	0.5208
No log	7.2	108	0.5030	0.5053	0.5018
No log	7.3333	110	0.5677	0.4910	0.5661
No log	7.4667	112	0.6657	0.4587	0.6638
No log	7.6	114	0.6913	0.4396	0.6893
No log	7.7333	116	0.6322	0.4662	0.6303
No log	7.8667	118	0.5615	0.4847	0.5599
No log	8.0	120	0.5037	0.5192	0.5025
No log	8.1333	122	0.4986	0.5275	0.4974
No log	8.2667	124	0.5375	0.4975	0.5361
No log	8.4	126	0.6271	0.4694	0.6252
No log	8.5333	128	0.7181	0.4380	0.7160
No log	8.6667	130	0.7262	0.4320	0.7241
No log	8.8	132	0.6675	0.4566	0.6655
No log	8.9333	134	0.5875	0.4826	0.5859
No log	9.0667	136	0.5616	0.4930	0.5602
No log	9.2	138	0.5683	0.4914	0.5668
No log	9.3333	140	0.5908	0.4826	0.5892
No log	9.4667	142	0.6226	0.4814	0.6208
No log	9.6	144	0.6525	0.4656	0.6507
No log	9.7333	146	0.6742	0.4610	0.6723
No log	9.8667	148	0.6879	0.4652	0.6859
No log	10.0	150	0.6885	0.4652	0.6865

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1