distilbert-base-uncased-distilled-clinc-own-best-run

This model is a fine-tuned version of distilbert-base-uncased on the clinc_oos dataset. It achieves the following results on the evaluation set:

• Loss: 0.0833
• Accuracy: 0.9487

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 48
• eval_batch_size: 48
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.9149	1.0	318	0.5756	0.7513
0.4412	2.0	636	0.2713	0.8881
0.2366	3.0	954	0.1653	0.9229
0.1601	4.0	1272	0.1283	0.9303
0.1265	5.0	1590	0.1121	0.9365
0.1096	6.0	1908	0.1033	0.9423
0.0991	7.0	2226	0.0976	0.9406
0.0923	8.0	2544	0.0943	0.9410
0.0879	9.0	2862	0.0919	0.9445
0.0842	10.0	3180	0.0899	0.9471
0.0815	11.0	3498	0.0883	0.9461
0.0792	12.0	3816	0.0871	0.9484
0.0775	13.0	4134	0.0868	0.9458
0.0758	14.0	4452	0.0859	0.9474
0.0747	15.0	4770	0.0855	0.9487
0.0738	16.0	5088	0.0847	0.9474
0.0728	17.0	5406	0.0850	0.9465
0.0721	18.0	5724	0.0845	0.9481
0.0714	19.0	6042	0.0842	0.9484
0.0706	20.0	6360	0.0836	0.9497
0.0704	21.0	6678	0.0838	0.9490
0.0701	22.0	6996	0.0831	0.9484
0.0697	23.0	7314	0.0834	0.9490
0.0696	24.0	7632	0.0834	0.9484
0.0693	25.0	7950	0.0833	0.9487

Framework versions

• Transformers 4.16.2
• Pytorch 2.4.1+cu121
• Datasets 1.16.1
• Tokenizers 0.19.1