m-minilm-l12-h384-mal-fake-news-detection-finetune

This model is a fine-tuned version of microsoft/Multilingual-MiniLM-L12-H384 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4168
• Accuracy: 0.8442
• F1: 0.8374

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 9

Training results

Training Loss	Epoch	Step	Accuracy	F1	Validation Loss
0.6928	0.3846	5	0.6233	0.4428	0.6868
0.6747	0.7692	10	0.6945	0.6631	0.6368
0.5988	1.1538	15	0.7399	0.7225	0.5638
0.5515	1.5385	20	0.7252	0.7627	0.5705
0.5011	1.9231	25	0.7939	0.7857	0.4777
0.4728	2.3077	30	0.7362	0.7691	0.5495
0.4543	2.6923	35	0.8037	0.7985	0.4529
0.4103	3.0769	40	0.8037	0.7975	0.4326
0.3934	3.4615	45	0.8049	0.8073	0.4277
0.3582	3.8462	50	0.8098	0.8112	0.4283
0.332	4.2308	55	0.8282	0.8219	0.4182
0.318	4.6154	60	0.8110	0.8122	0.4175
0.3026	5.0	65	0.8319	0.8237	0.4094
0.2861	5.3846	70	0.8221	0.8230	0.4154
0.3015	5.7692	75	0.8294	0.8260	0.4117
0.2569	6.1538	80	0.4142	0.8282	0.8254
0.266	6.5385	85	0.4359	0.8074	0.8115
0.2758	6.9231	90	0.4168	0.8442	0.8374
0.2551	7.3077	95	0.4181	0.8270	0.8261
0.2404	7.6923	100	0.4194	0.8307	0.8296
0.2226	8.0769	105	0.4174	0.8368	0.8301
0.2213	8.4615	110	0.4253	0.8184	0.8200
0.222	8.8462	115	0.4227	0.8282	0.8276

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1+cu121
• Datasets 3.2.0
• Tokenizers 0.20.3