arabert_cross_relevance_task3_fold6

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3958
• Qwk: 0.2096
• Mse: 0.3959

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1176	2	0.7510	0.0098	0.7490
No log	0.2353	4	0.3906	0.1162	0.3902
No log	0.3529	6	0.3053	0.1347	0.3057
No log	0.4706	8	0.2972	0.1520	0.2976
No log	0.5882	10	0.2828	0.1830	0.2831
No log	0.7059	12	0.2698	0.1906	0.2703
No log	0.8235	14	0.2743	0.1500	0.2750
No log	0.9412	16	0.2742	0.1578	0.2747
No log	1.0588	18	0.3079	0.1726	0.3083
No log	1.1765	20	0.3022	0.1999	0.3025
No log	1.2941	22	0.2720	0.2049	0.2724
No log	1.4118	24	0.2712	0.2178	0.2717
No log	1.5294	26	0.2820	0.2018	0.2823
No log	1.6471	28	0.2988	0.2137	0.2990
No log	1.7647	30	0.2801	0.2099	0.2803
No log	1.8824	32	0.2536	0.2155	0.2541
No log	2.0	34	0.2547	0.2222	0.2553
No log	2.1176	36	0.2817	0.1893	0.2821
No log	2.2353	38	0.3173	0.2127	0.3175
No log	2.3529	40	0.2805	0.1991	0.2808
No log	2.4706	42	0.2617	0.2112	0.2624
No log	2.5882	44	0.2662	0.2183	0.2670
No log	2.7059	46	0.2560	0.2109	0.2565
No log	2.8235	48	0.3002	0.2083	0.3005
No log	2.9412	50	0.3062	0.2127	0.3065
No log	3.0588	52	0.2753	0.1995	0.2757
No log	3.1765	54	0.2479	0.2057	0.2485
No log	3.2941	56	0.2470	0.1860	0.2476
No log	3.4118	58	0.2610	0.2099	0.2614
No log	3.5294	60	0.2690	0.2044	0.2693
No log	3.6471	62	0.2641	0.2067	0.2645
No log	3.7647	64	0.2712	0.1995	0.2715
No log	3.8824	66	0.2789	0.1943	0.2792
No log	4.0	68	0.2918	0.2127	0.2920
No log	4.1176	70	0.2717	0.2038	0.2720
No log	4.2353	72	0.2866	0.2127	0.2868
No log	4.3529	74	0.2850	0.2127	0.2852
No log	4.4706	76	0.2771	0.2292	0.2774
No log	4.5882	78	0.2687	0.1915	0.2690
No log	4.7059	80	0.2758	0.2044	0.2761
No log	4.8235	82	0.3101	0.2211	0.3103
No log	4.9412	84	0.3572	0.2110	0.3573
No log	5.0588	86	0.3906	0.1858	0.3906
No log	5.1765	88	0.3591	0.1982	0.3592
No log	5.2941	90	0.3170	0.2140	0.3173
No log	5.4118	92	0.3098	0.2140	0.3102
No log	5.5294	94	0.3073	0.2048	0.3077
No log	5.6471	96	0.3304	0.2001	0.3307
No log	5.7647	98	0.3825	0.2103	0.3825
No log	5.8824	100	0.3978	0.1979	0.3978
No log	6.0	102	0.3627	0.2110	0.3629
No log	6.1176	104	0.3352	0.1985	0.3354
No log	6.2353	106	0.3174	0.2127	0.3177
No log	6.3529	108	0.3418	0.2070	0.3421
No log	6.4706	110	0.3437	0.1982	0.3439
No log	6.5882	112	0.3452	0.1982	0.3454
No log	6.7059	114	0.3485	0.2070	0.3487
No log	6.8235	116	0.3713	0.1979	0.3714
No log	6.9412	118	0.3854	0.1858	0.3855
No log	7.0588	120	0.3950	0.1821	0.3949
No log	7.1765	122	0.3832	0.1858	0.3832
No log	7.2941	124	0.3695	0.2019	0.3697
No log	7.4118	126	0.3763	0.2058	0.3764
No log	7.5294	128	0.3884	0.1898	0.3885
No log	7.6471	130	0.4063	0.1785	0.4064
No log	7.7647	132	0.3900	0.1780	0.3901
No log	7.8824	134	0.3822	0.1739	0.3823
No log	8.0	136	0.3725	0.1856	0.3726
No log	8.1176	138	0.3559	0.1854	0.3560
No log	8.2353	140	0.3576	0.2023	0.3578
No log	8.3529	142	0.3825	0.1977	0.3826
No log	8.4706	144	0.3960	0.2015	0.3961
No log	8.5882	146	0.3877	0.2015	0.3877
No log	8.7059	148	0.3846	0.2096	0.3847
No log	8.8235	150	0.3711	0.2058	0.3712
No log	8.9412	152	0.3592	0.2103	0.3594
No log	9.0588	154	0.3526	0.2103	0.3529
No log	9.1765	156	0.3597	0.2141	0.3598
No log	9.2941	158	0.3711	0.2058	0.3712
No log	9.4118	160	0.3827	0.2096	0.3828
No log	9.5294	162	0.3867	0.2096	0.3867
No log	9.6471	164	0.3912	0.2096	0.3912
No log	9.7647	166	0.3942	0.2096	0.3942
No log	9.8824	168	0.3960	0.2096	0.3960
No log	10.0	170	0.3958	0.2096	0.3959

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1