jehone-shqip

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0986
• Wer: 50.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.2919	0.31	1	1.0503	57.3333
0.4118	0.62	2	1.0144	56.6667
0.2816	0.92	3	0.9834	55.3333
0.1495	1.23	4	0.9662	53.3333
0.1293	1.54	5	0.9698	50.0
0.0617	1.85	6	0.9868	54.0
0.0802	2.15	7	1.0159	55.3333
0.0519	2.46	8	1.0413	56.6667
0.0367	2.77	9	1.0518	58.0000
0.0304	3.08	10	1.0545	57.3333
0.022	3.38	11	1.0491	58.6667
0.0169	3.69	12	1.0441	56.6667
0.014	4.0	13	1.0361	57.3333
0.0106	4.31	14	1.0315	56.0000
0.0079	4.62	15	1.0294	58.0000
0.0081	4.92	16	1.0300	56.6667
0.006	5.23	17	1.0319	56.6667
0.006	5.54	18	1.0353	56.6667
0.0048	5.85	19	1.0398	57.3333
0.0045	6.15	20	1.0453	56.0000
0.0041	6.46	21	1.0506	55.3333
0.0039	6.77	22	1.0558	55.3333
0.004	7.08	23	1.0613	54.0
0.0033	7.38	24	1.0663	52.6667
0.0034	7.69	25	1.0706	52.6667
0.0031	8.0	26	1.0751	52.0
0.003	8.31	27	1.0791	52.0
0.0028	8.62	28	1.0828	51.3333
0.0026	8.92	29	1.0861	50.6667
0.003	9.23	30	1.0890	50.6667
0.0026	9.54	31	1.0912	50.6667
0.0022	9.85	32	1.0928	50.0
0.0024	10.15	33	1.0944	50.0
0.0023	10.46	34	1.0957	50.0
0.0024	10.77	35	1.0966	50.0
0.0024	11.08	36	1.0974	50.0
0.0023	11.38	37	1.0979	50.0
0.0023	11.69	38	1.0983	50.0
0.0023	12.0	39	1.0983	50.0
0.002	12.31	40	1.0986	50.0

Framework versions

• Transformers 4.40.0.dev0
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2