distilbert-base-uncased_fold_10_binary_v1

This model is a fine-tuned version of distilbert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6912
• F1: 0.7977

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	F1
No log	1.0	288	0.4002	0.8012
0.4056	2.0	576	0.4372	0.8075
0.4056	3.0	864	0.4720	0.8071
0.1958	4.0	1152	0.8156	0.7980
0.1958	5.0	1440	0.8633	0.8055
0.0847	6.0	1728	0.9761	0.8041
0.0356	7.0	2016	1.1816	0.7861
0.0356	8.0	2304	1.2251	0.7918
0.0215	9.0	2592	1.3423	0.7798
0.0215	10.0	2880	1.3888	0.7913
0.013	11.0	3168	1.2899	0.8040
0.013	12.0	3456	1.4247	0.8051
0.0049	13.0	3744	1.5436	0.7991
0.0061	14.0	4032	1.5762	0.7991
0.0061	15.0	4320	1.5461	0.7998
0.0054	16.0	4608	1.5622	0.8018
0.0054	17.0	4896	1.6658	0.7991
0.0021	18.0	5184	1.6765	0.7972
0.0021	19.0	5472	1.6864	0.7973
0.0052	20.0	5760	1.6303	0.8030
0.0029	21.0	6048	1.6631	0.7947
0.0029	22.0	6336	1.6571	0.8006
0.0027	23.0	6624	1.6729	0.7949
0.0027	24.0	6912	1.6931	0.7934
0.0001	25.0	7200	1.6912	0.7977

Framework versions

• Transformers 4.21.0
• Pytorch 1.12.0+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1