bert-base-uncased-finetuned-swag

This model is a fine-tuned version of Yama/bert-base-uncased-finetuned-swag on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0061
• Accuracy: 0.9958

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	150	1.3780	0.3592
No log	2.0	300	1.3234	0.4383
No log	3.0	450	1.3158	0.4992
1.3577	4.0	600	1.1356	0.5792
1.3577	5.0	750	0.7939	0.7217
1.3577	6.0	900	0.6167	0.7958
1.0479	7.0	1050	0.4737	0.8467
1.0479	8.0	1200	0.3424	0.8867
1.0479	9.0	1350	0.2448	0.9142
0.5968	10.0	1500	0.2117	0.9158
0.5968	11.0	1650	0.1589	0.9467
0.5968	12.0	1800	0.1420	0.9492
0.5968	13.0	1950	0.0970	0.9675
0.3341	14.0	2100	0.1014	0.9725
0.3341	15.0	2250	0.0678	0.9742
0.3341	16.0	2400	0.0624	0.9825
0.1802	17.0	2550	0.0407	0.9783
0.1802	18.0	2700	0.0501	0.9858
0.1802	19.0	2850	0.0341	0.9867
0.1213	20.0	3000	0.0284	0.9883
0.1213	21.0	3150	0.0398	0.9883
0.1213	22.0	3300	0.0290	0.9908
0.1213	23.0	3450	0.0211	0.9908
0.0758	24.0	3600	0.0179	0.9908
0.0758	25.0	3750	0.0151	0.9917
0.0758	26.0	3900	0.0154	0.9933
0.0464	27.0	4050	0.0216	0.9942
0.0464	28.0	4200	0.0124	0.9942
0.0464	29.0	4350	0.0122	0.9942
0.0306	30.0	4500	0.0103	0.9942
0.0306	31.0	4650	0.0094	0.9942
0.0306	32.0	4800	0.0083	0.9942
0.0306	33.0	4950	0.0079	0.9958
0.0201	34.0	5100	0.0079	0.9950
0.0201	35.0	5250	0.0069	0.9958
0.0201	36.0	5400	0.0069	0.9950
0.0205	37.0	5550	0.0060	0.9967
0.0205	38.0	5700	0.0060	0.9958
0.0205	39.0	5850	0.0061	0.9958
0.0102	40.0	6000	0.0061	0.9958

Framework versions

• Transformers 4.35.0
• Pytorch 2.1.0+cu118
• Datasets 2.14.6
• Tokenizers 0.14.1