xlm-roberta-base-hau-finetuned-augmentation-LUNAR

This model is a fine-tuned version of FacebookAI/xlm-roberta-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3543
• F1: 0.6620
• Roc Auc: 0.7803
• Accuracy: 0.4983

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 100
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	F1	Roc Auc	Accuracy
0.4603	1.0	144	0.4533	0.0933	0.5307	0.1916
0.4368	2.0	288	0.4219	0.1953	0.5812	0.2003
0.3826	3.0	432	0.3880	0.3288	0.6189	0.2787
0.3535	4.0	576	0.3562	0.5208	0.6918	0.3833
0.3018	5.0	720	0.3511	0.5697	0.7308	0.4007
0.2543	6.0	864	0.3593	0.6030	0.7492	0.4338
0.2549	7.0	1008	0.3463	0.6063	0.7436	0.4460
0.2183	8.0	1152	0.3425	0.6112	0.7484	0.4477
0.195	9.0	1296	0.3502	0.6004	0.7432	0.4355
0.1842	10.0	1440	0.3358	0.6223	0.7506	0.4686
0.151	11.0	1584	0.3451	0.6245	0.7578	0.4669
0.1385	12.0	1728	0.3383	0.6313	0.7584	0.4669
0.1286	13.0	1872	0.3492	0.6445	0.7692	0.4739
0.115	14.0	2016	0.3502	0.6553	0.7753	0.4913
0.1031	15.0	2160	0.3516	0.6529	0.7771	0.4826
0.1068	16.0	2304	0.3529	0.6448	0.7685	0.4808
0.0818	17.0	2448	0.3522	0.6542	0.7741	0.4895
0.0906	18.0	2592	0.3543	0.6620	0.7803	0.4983
0.0878	19.0	2736	0.3541	0.6595	0.7780	0.4948
0.0872	20.0	2880	0.3545	0.6584	0.7777	0.4948

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.0
• Datasets 3.0.1
• Tokenizers 0.20.0