MeMo_BERT-SA_danbert

This model is a fine-tuned version of alexanderfalk/danbert-small-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.4690
• F1-score: 0.6544

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	F1-score
No log	1.0	297	1.0065	0.5054
0.8685	2.0	594	1.0551	0.5964
0.8685	3.0	891	1.3189	0.5761
0.3832	4.0	1188	2.0270	0.6322
0.3832	5.0	1485	2.1568	0.6076
0.1519	6.0	1782	3.3066	0.5763
0.0403	7.0	2079	3.1085	0.6049
0.0403	8.0	2376	3.1069	0.6269
0.009	9.0	2673	3.2610	0.6213
0.009	10.0	2970	3.3529	0.6355
0.0123	11.0	3267	3.4172	0.6306
0.0089	12.0	3564	3.4950	0.6187
0.0089	13.0	3861	3.7618	0.6117
0.0053	14.0	4158	3.5352	0.6252
0.0053	15.0	4455	3.7667	0.6120
0.0047	16.0	4752	3.4690	0.6544
0.0059	17.0	5049	3.6646	0.6256
0.0059	18.0	5346	3.6611	0.6319
0.0018	19.0	5643	3.7555	0.6423
0.0018	20.0	5940	3.7564	0.6422

Framework versions

• Transformers 4.38.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2