wav2vec2-xls-r-1b-E3-faroese-100h-30-epochs_20250124_v2

This model is a fine-tuned version of davidilag/wav2vec2-xls-r-1b-scandinavian-E3-100h-30-epochs-20250123 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1056
• Wer: 19.0378
• Cer: 4.1422

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 5000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.7255	0.4877	1000	0.5000	56.2453	16.3636
0.533	0.9754	2000	0.3085	40.8776	10.8415
0.4289	1.4628	3000	0.2684	36.2206	9.7124
0.3957	1.9505	4000	0.2505	34.8460	9.2698
0.3304	2.4379	5000	0.2366	32.9207	8.5132
0.3429	2.9256	6000	0.2209	32.3215	8.3301
0.318	3.4131	7000	0.2141	31.1187	8.0477
0.2793	3.9008	8000	0.2012	29.7484	7.5853
0.2599	4.3882	9000	0.1913	28.8981	7.3084
0.2542	4.8759	10000	0.1754	27.9993	7.0196
0.2306	5.3633	11000	0.1723	27.1930	6.8160
0.2186	5.8510	12000	0.1687	26.6423	6.6259
0.191	6.3385	13000	0.1649	26.8450	6.5880
0.1837	6.8261	14000	0.1634	26.6203	6.6212
0.181	7.3136	15000	0.1500	25.8492	6.2724
0.1655	7.8013	16000	0.1423	25.1002	5.9695
0.1458	8.2887	17000	0.1555	25.4527	6.1699
0.1519	8.7764	18000	0.1448	25.1487	6.0665
0.1296	9.2638	19000	0.1538	25.0121	6.0555
0.1335	9.7515	20000	0.1484	24.5539	5.8622
0.1193	10.2390	21000	0.1450	24.4570	5.8637
0.1113	10.7267	22000	0.1322	23.5802	5.5174
0.1236	11.2141	23000	0.1420	23.7388	5.6760
0.1128	11.7018	24000	0.1259	23.3026	5.3580
0.1028	12.1892	25000	0.1368	23.3731	5.4913
0.0934	12.6769	26000	0.1272	22.4655	5.2381
0.083	13.1644	27000	0.1361	22.5404	5.2633
0.0913	13.6520	28000	0.1316	22.6814	5.3044
0.0766	14.1395	29000	0.1337	22.0337	5.0850
0.0843	14.6272	30000	0.1220	22.2056	5.1434
0.079	15.1146	31000	0.1240	21.9192	5.0479
0.0759	15.6023	32000	0.1248	21.8046	4.9951
0.071	16.0897	33000	0.1262	21.8002	5.0101
0.0691	16.5774	34000	0.1164	21.5227	4.8704
0.0656	17.0649	35000	0.1187	21.3464	4.8057
0.0629	17.5525	36000	0.1132	21.2143	4.7939
0.0513	18.0400	37000	0.1215	21.0821	4.8010
0.0515	18.5277	38000	0.1147	20.7516	4.6937
0.0507	19.0151	39000	0.1131	20.7384	4.6716
0.0466	19.5028	40000	0.1173	20.5137	4.6116
0.0449	19.9905	41000	0.1106	20.3551	4.5588
0.0503	20.4779	42000	0.1127	20.2406	4.5114
0.0474	20.9656	43000	0.1084	20.1040	4.4483
0.0378	21.4531	44000	0.1080	19.9321	4.4167
0.0472	21.9407	45000	0.1070	19.8308	4.3868
0.033	22.4282	46000	0.1114	19.8528	4.3741
0.0361	22.9159	47000	0.1084	19.5048	4.3126
0.0363	23.4033	48000	0.1103	19.5048	4.2818
0.036	23.8910	49000	0.1043	19.3506	4.2621
0.0352	24.3784	50000	0.1047	19.4607	4.2763
0.029	24.8661	51000	0.1047	19.3594	4.2266
0.0343	25.3536	52000	0.1052	19.2272	4.2037
0.0322	25.8413	53000	0.1028	19.2581	4.2013
0.0296	26.3287	54000	0.1029	19.1303	4.1556
0.0262	26.8164	55000	0.1054	19.1523	4.1698
0.0251	27.3038	56000	0.1053	19.1303	4.1540
0.0305	27.7915	57000	0.1060	19.0906	4.1477
0.0303	28.2790	58000	0.1057	19.0289	4.1367
0.0308	28.7666	59000	0.1060	19.0334	4.1453
0.0382	29.2541	60000	0.1057	19.0289	4.1438
0.037	29.7418	61000	0.1056	19.0378	4.1422

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0