arabert_cross_relevance_task4_fold2

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4752
• Qwk: 0.1093
• Mse: 0.4759

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1111	2	0.5785	0.0868	0.5777
No log	0.2222	4	0.2943	-0.0164	0.2947
No log	0.3333	6	0.3987	0.0024	0.3993
No log	0.4444	8	0.3899	0.0752	0.3904
No log	0.5556	10	0.3473	-0.0042	0.3477
No log	0.6667	12	0.4204	0.0370	0.4209
No log	0.7778	14	0.5631	-0.0284	0.5638
No log	0.8889	16	0.5554	-0.0028	0.5561
No log	1.0	18	0.5578	0.0426	0.5584
No log	1.1111	20	0.4455	0.0325	0.4461
No log	1.2222	22	0.3963	0.0368	0.3969
No log	1.3333	24	0.3919	0.0243	0.3924
No log	1.4444	26	0.4089	0.0665	0.4094
No log	1.5556	28	0.3918	0.0619	0.3923
No log	1.6667	30	0.4355	-0.0079	0.4360
No log	1.7778	32	0.4965	0.1166	0.4971
No log	1.8889	34	0.5320	-0.0186	0.5326
No log	2.0	36	0.5879	0.0536	0.5885
No log	2.1111	38	0.5920	0.1117	0.5926
No log	2.2222	40	0.5586	0.1740	0.5592
No log	2.3333	42	0.4887	0.0570	0.4892
No log	2.4444	44	0.4523	0.0512	0.4527
No log	2.5556	46	0.4295	0.0633	0.4299
No log	2.6667	48	0.4439	0.0643	0.4444
No log	2.7778	50	0.4579	-0.0142	0.4585
No log	2.8889	52	0.4404	0.0594	0.4409
No log	3.0	54	0.4585	0.0325	0.4590
No log	3.1111	56	0.5216	0.1088	0.5222
No log	3.2222	58	0.5552	0.0747	0.5559
No log	3.3333	60	0.5458	0.1373	0.5465
No log	3.4444	62	0.4947	0.1774	0.4953
No log	3.5556	64	0.4605	0.0419	0.4611
No log	3.6667	66	0.4341	-0.0514	0.4346
No log	3.7778	68	0.4227	-0.0500	0.4232
No log	3.8889	70	0.4149	-0.0016	0.4154
No log	4.0	72	0.4133	0.0112	0.4138
No log	4.1111	74	0.4299	0.0681	0.4304
No log	4.2222	76	0.4431	0.0874	0.4437
No log	4.3333	78	0.4609	0.1618	0.4615
No log	4.4444	80	0.4845	0.1449	0.4851
No log	4.5556	82	0.5171	0.1801	0.5178
No log	4.6667	84	0.5429	0.0638	0.5436
No log	4.7778	86	0.5295	0.1054	0.5302
No log	4.8889	88	0.4873	0.1361	0.4880
No log	5.0	90	0.4385	0.0551	0.4391
No log	5.1111	92	0.4189	0.0586	0.4195
No log	5.2222	94	0.4360	0.0730	0.4366
No log	5.3333	96	0.4771	0.0422	0.4778
No log	5.4444	98	0.4940	0.0473	0.4947
No log	5.5556	100	0.5040	0.0525	0.5047
No log	5.6667	102	0.4813	0.0030	0.4819
No log	5.7778	104	0.4381	0.0917	0.4387
No log	5.8889	106	0.3938	-0.0553	0.3943
No log	6.0	108	0.3964	-0.0475	0.3969
No log	6.1111	110	0.4237	0.0059	0.4242
No log	6.2222	112	0.4695	0.1283	0.4701
No log	6.3333	114	0.4933	0.0923	0.4940
No log	6.4444	116	0.5089	0.0578	0.5096
No log	6.5556	118	0.5097	0.0886	0.5105
No log	6.6667	120	0.4867	0.0724	0.4874
No log	6.7778	122	0.4496	0.0831	0.4502
No log	6.8889	124	0.4192	0.0197	0.4198
No log	7.0	126	0.4075	0.0496	0.4081
No log	7.1111	128	0.4043	0.0369	0.4049
No log	7.2222	130	0.4164	0.0326	0.4170
No log	7.3333	132	0.4361	0.0815	0.4367
No log	7.4444	134	0.4517	0.1068	0.4524
No log	7.5556	136	0.4513	0.1209	0.4519
No log	7.6667	138	0.4452	0.1056	0.4458
No log	7.7778	140	0.4514	0.0743	0.4520
No log	7.8889	142	0.4671	0.1472	0.4677
No log	8.0	144	0.4902	0.0969	0.4909
No log	8.1111	146	0.5000	-0.0230	0.5007
No log	8.2222	148	0.4999	-0.0028	0.5006
No log	8.3333	150	0.4924	0.0374	0.4931
No log	8.4444	152	0.4849	0.0857	0.4855
No log	8.5556	154	0.4790	0.0857	0.4796
No log	8.6667	156	0.4774	0.0857	0.4780
No log	8.7778	158	0.4814	0.1004	0.4820
No log	8.8889	160	0.4916	0.0866	0.4923
No log	9.0	162	0.4998	0.0525	0.5005
No log	9.1111	164	0.5009	0.0676	0.5016
No log	9.2222	166	0.4971	0.0676	0.4978
No log	9.3333	168	0.4903	0.1166	0.4910
No log	9.4444	170	0.4831	0.1641	0.4838
No log	9.5556	172	0.4811	0.1832	0.4818
No log	9.6667	174	0.4780	0.1534	0.4786
No log	9.7778	176	0.4767	0.1239	0.4774
No log	9.8889	178	0.4754	0.1093	0.4760
No log	10.0	180	0.4752	0.1093	0.4759

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1