bert_base_for_whole_train_result_Spam-Ham_farshad_4_2

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0511
• Accuracy: 0.9939
• F1: 0.9941

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 64
• total_train_batch_size: 4096
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.6361	2.9250	50	0.4585	0.8883	0.8874
0.2472	5.8501	100	0.0880	0.9722	0.9728
0.0507	8.7751	150	0.0429	0.9861	0.9865
0.0223	11.7002	200	0.0454	0.9849	0.9853
0.0121	14.6252	250	0.0397	0.9896	0.9899
0.0075	17.5503	300	0.0397	0.9916	0.9919
0.006	20.4753	350	0.0388	0.9919	0.9922
0.005	23.4004	400	0.0363	0.9925	0.9927
0.0034	26.3254	450	0.0404	0.9904	0.9908
0.0044	29.2505	500	0.0349	0.9925	0.9927
0.0016	32.1755	550	0.0456	0.9904	0.9907
0.0015	35.1005	600	0.0582	0.9878	0.9881
0.0022	38.0256	650	0.0854	0.9846	0.9850
0.0018	40.9506	700	0.0423	0.9933	0.9936
0.001	43.8757	750	0.0557	0.9907	0.9910
0.0009	46.8007	800	0.0490	0.9925	0.9927
0.001	49.7258	850	0.0565	0.9904	0.9907
0.0006	52.6508	900	0.0602	0.9910	0.9913
0.0021	55.5759	950	0.0482	0.9907	0.9910
0.0008	58.5009	1000	0.0619	0.9893	0.9896
0.0008	61.4260	1050	0.0476	0.9919	0.9922
0.0007	64.3510	1100	0.0452	0.9927	0.9930
0.0005	67.2761	1150	0.0468	0.9939	0.9941
0.0005	70.2011	1200	0.0546	0.9916	0.9919
0.0005	73.1261	1250	0.0525	0.9919	0.9922
0.0004	76.0512	1300	0.0498	0.9936	0.9938
0.0004	78.9762	1350	0.0655	0.9896	0.9899
0.0004	81.9013	1400	0.0524	0.9922	0.9924
0.0005	84.8263	1450	0.0524	0.9916	0.9919
0.0004	87.7514	1500	0.0516	0.9927	0.9930
0.0004	90.6764	1550	0.0522	0.9927	0.9930
0.0004	93.6015	1600	0.0512	0.9939	0.9941
0.0003	96.5265	1650	0.0511	0.9939	0.9941

Framework versions

• Transformers 4.40.0
• Pytorch 2.4.1+cu121
• Datasets 3.1.0
• Tokenizers 0.19.1