distilbert-base-uncased_fold_3_binary

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.8310
• F1: 0.7584

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	F1
No log	1.0	289	0.4560	0.7522
0.4008	2.0	578	0.4790	0.7567
0.4008	3.0	867	0.6368	0.7557
0.1967	4.0	1156	0.6854	0.7534
0.1967	5.0	1445	0.9823	0.7309
0.0768	6.0	1734	1.2531	0.7511
0.0202	7.0	2023	1.2906	0.7391
0.0202	8.0	2312	1.4025	0.7460
0.0087	9.0	2601	1.5713	0.7507
0.0087	10.0	2890	1.4212	0.7528
0.0162	11.0	3179	1.5775	0.7511
0.0162	12.0	3468	1.6361	0.7377
0.0048	13.0	3757	1.6972	0.7542
0.0098	14.0	4046	1.6569	0.7565
0.0098	15.0	4335	1.7547	0.7325
0.0042	16.0	4624	1.8108	0.7544
0.0042	17.0	4913	1.7593	0.7554
0.0041	18.0	5202	1.7582	0.7551
0.0041	19.0	5491	1.8200	0.7512
0.0029	20.0	5780	1.8310	0.7584
0.0018	21.0	6069	1.8146	0.7568
0.0018	22.0	6358	1.7870	0.7558
0.0029	23.0	6647	1.8508	0.7530
0.0029	24.0	6936	1.8327	0.7543
0.0001	25.0	7225	1.8546	0.7561

Framework versions

• Transformers 4.21.0
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1