Banglabert_nwp_finetuning_test1

This model is a fine-tuned version of csebuetnlp/banglabert on the None dataset. It achieves the following results on the evaluation set:

• Loss: 4.1145

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	403	7.4663
7.9712	2.0	806	6.5917
6.8071	3.0	1209	6.0256
6.0374	4.0	1612	5.6658
5.6471	5.0	2015	5.4585
5.6471	6.0	2418	5.2872
5.3912	7.0	2821	5.2068
5.1951	8.0	3224	4.9943
4.9863	9.0	3627	4.9361
4.8637	10.0	4030	5.0372
4.8637	11.0	4433	4.8487
4.7391	12.0	4836	4.7716
4.6406	13.0	5239	4.6486
4.5531	14.0	5642	4.6206
4.438	15.0	6045	4.5695
4.438	16.0	6448	4.5419
4.349	17.0	6851	4.5311
4.2898	18.0	7254	4.5808
4.215	19.0	7657	4.4420
4.1477	20.0	8060	4.4000
4.1477	21.0	8463	4.4062
4.0975	22.0	8866	4.4725
4.0181	23.0	9269	4.3565
3.9865	24.0	9672	4.2207
3.9196	25.0	10075	4.2460
3.9196	26.0	10478	4.2876
3.9069	27.0	10881	4.2645
3.8349	28.0	11284	4.2974
3.7824	29.0	11687	4.3730
3.736	30.0	12090	4.2604
3.736	31.0	12493	4.3195
3.7202	32.0	12896	4.1342
3.6897	33.0	13299	4.3159
3.6388	34.0	13702	4.1019
3.6448	35.0	14105	4.1839
3.6099	36.0	14508	4.1900
3.6099	37.0	14911	4.0979
3.5708	38.0	15314	4.1737
3.5502	39.0	15717	4.1915
3.5159	40.0	16120	4.1158
3.5086	41.0	16523	4.0181
3.5086	42.0	16926	4.1328
3.4808	43.0	17329	4.0247
3.4733	44.0	17732	4.1688
3.4681	45.0	18135	4.1177
3.4689	46.0	18538	4.1093
3.4689	47.0	18941	4.1418
3.434	48.0	19344	4.0152
3.4513	49.0	19747	4.1412
3.4158	50.0	20150	4.1036

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3