Neuria_BERT_Contexto_0108

This model is a fine-tuned version of dccuchile/bert-base-spanish-wwm-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0851
• F1 Micro: 0.8428
• F1 Macro: 0.5221

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Micro	F1 Macro
0.4708	0.96	18	0.3215	0.0	0.0
0.3044	1.9733	37	0.2748	0.0230	0.0096
0.2664	2.9867	56	0.2490	0.3301	0.0883
0.2376	4.0	75	0.2225	0.3619	0.1010
0.2196	4.96	93	0.1997	0.5254	0.1891
0.1815	5.9733	112	0.1802	0.6190	0.2344
0.1592	6.9867	131	0.1655	0.6032	0.2441
0.1362	8.0	150	0.1492	0.7059	0.3614
0.126	8.96	168	0.1383	0.7234	0.4036
0.1054	9.9733	187	0.1311	0.7639	0.4380
0.095	10.9867	206	0.1291	0.7639	0.4369
0.0858	12.0	225	0.1195	0.7891	0.4683
0.0816	12.96	243	0.1179	0.7974	0.4815
0.0707	13.9733	262	0.1080	0.8105	0.4927
0.0655	14.9867	281	0.1074	0.8129	0.4962
0.0609	16.0	300	0.1041	0.8333	0.5166
0.0599	16.96	318	0.1011	0.8258	0.5037
0.0537	17.9733	337	0.0988	0.8235	0.4994
0.0512	18.9867	356	0.0976	0.8258	0.5115
0.0485	20.0	375	0.0965	0.8153	0.5075
0.0491	20.96	393	0.0945	0.8333	0.5181
0.0447	21.9733	412	0.0939	0.8375	0.5102
0.0426	22.9867	431	0.0949	0.8258	0.5010
0.0418	24.0	450	0.0926	0.8447	0.5247
0.0423	24.96	468	0.0929	0.8375	0.5102
0.0389	25.9733	487	0.0920	0.85	0.5331
0.0375	26.9867	506	0.0921	0.8462	0.5246
0.0368	28.0	525	0.0900	0.8101	0.4962
0.0375	28.96	543	0.0914	0.8408	0.5125
0.0349	29.9733	562	0.0894	0.8481	0.5243
0.034	30.9867	581	0.0887	0.8447	0.5235
0.0334	32.0	600	0.0871	0.8428	0.5221
0.0342	32.96	618	0.0863	0.8354	0.5184
0.0317	33.9733	637	0.0875	0.8280	0.5052
0.0311	34.9867	656	0.0877	0.8354	0.5089
0.0307	36.0	675	0.0874	0.8354	0.5184
0.0318	36.96	693	0.0863	0.8428	0.5221
0.0297	37.9733	712	0.0854	0.8280	0.5145
0.0294	38.9867	731	0.0867	0.8375	0.5200
0.0292	40.0	750	0.0856	0.8428	0.5221
0.0306	40.96	768	0.0857	0.8354	0.5184
0.0287	41.9733	787	0.0856	0.8428	0.5221
0.0284	42.9867	806	0.0847	0.8354	0.5184
0.0284	44.0	825	0.0849	0.8428	0.5221
0.0296	44.96	843	0.0854	0.8428	0.5221
0.028	45.9733	862	0.0852	0.8428	0.5221
0.0278	46.9867	881	0.0850	0.8428	0.5221
0.0279	48.0	900	0.0851	0.8428	0.5221

Framework versions

• Transformers 4.44.1
• Pytorch 2.4.1
• Datasets 2.19.1
• Tokenizers 0.19.1