wav2vec2-xls-r-1b-E2-faroese-100h-30-epochs_20250124_v3

This model is a fine-tuned version of davidilag/wav2vec2-xls-r-1b-scandinavian-E2-100h-30-epochs-20250123 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1178
• Wer: 19.6766
• Cer: 4.2984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 5000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
3.0158	0.4877	1000	3.2867	100.1278	90.1077
0.8828	0.9754	2000	0.6226	64.6429	19.5448
0.5525	1.4628	3000	0.3515	42.8030	12.1094
0.4761	1.9505	4000	0.2862	37.6437	10.2765
0.3943	2.4379	5000	0.2760	35.6831	9.6572
0.3827	2.9256	6000	0.2715	34.3834	9.4094
0.3386	3.4131	7000	0.2373	32.3964	8.4524
0.2965	3.9008	8000	0.2105	30.3476	7.8638
0.2807	4.3882	9000	0.2004	29.8145	7.6445
0.2881	4.8759	10000	0.1956	29.4444	7.5324
0.2609	5.3633	11000	0.1880	28.4046	7.2800
0.2518	5.8510	12000	0.1842	28.1667	7.0835
0.2191	6.3385	13000	0.1790	27.5984	6.8847
0.2159	6.8261	14000	0.1894	28.2416	7.1790
0.2076	7.3136	15000	0.1714	26.7612	6.7245
0.2007	7.8013	16000	0.1805	27.0653	6.9486
0.1662	8.2887	17000	0.1792	26.1136	6.4886
0.1764	8.7764	18000	0.1626	26.3823	6.5478
0.1426	9.2638	19000	0.1623	25.3646	6.2456
0.1406	9.7515	20000	0.1642	25.4747	6.2275
0.144	10.2390	21000	0.1620	25.1002	6.1880
0.1328	10.7267	22000	0.1558	24.8227	6.0854
0.1366	11.2141	23000	0.1521	24.2235	5.9079
0.1223	11.7018	24000	0.1461	24.0913	5.7777
0.1195	12.1892	25000	0.1378	23.9855	5.7580
0.1218	12.6769	26000	0.1347	23.8137	5.5623
0.1069	13.1644	27000	0.1350	23.3159	5.5276
0.1037	13.6520	28000	0.1400	23.0735	5.4740
0.0885	14.1395	29000	0.1432	23.1528	5.4030
0.0934	14.6272	30000	0.1321	22.8841	5.3430
0.0836	15.1146	31000	0.1285	22.4303	5.1742
0.0875	15.6023	32000	0.1237	22.3422	5.1260
0.0741	16.0897	33000	0.1345	22.3333	5.2176
0.0712	16.5774	34000	0.1348	22.1439	5.0929
0.0747	17.0649	35000	0.1269	21.7650	4.9777
0.0735	17.5525	36000	0.1262	21.7782	4.9280
0.0598	18.0400	37000	0.1253	21.7518	4.9730
0.0564	18.5277	38000	0.1196	21.5095	4.9020
0.0574	19.0151	39000	0.1187	21.1438	4.7939
0.0526	19.5028	40000	0.1218	21.1394	4.7655
0.0495	19.9905	41000	0.1188	21.0116	4.6905
0.056	20.4779	42000	0.1160	20.7913	4.6392
0.0524	20.9656	43000	0.1180	20.7693	4.6140
0.0474	21.4531	44000	0.1211	20.5446	4.6100
0.054	21.9407	45000	0.1152	20.4036	4.5588
0.0339	22.4282	46000	0.1181	20.3595	4.4885
0.0437	22.9159	47000	0.1179	20.2273	4.4562
0.0408	23.4033	48000	0.1172	20.1524	4.4538
0.0418	23.8910	49000	0.1197	20.1260	4.4317
0.0415	24.3784	50000	0.1174	20.0423	4.4041
0.0353	24.8661	51000	0.1128	19.9410	4.3804
0.0385	25.3536	52000	0.1152	19.9674	4.3797
0.0365	25.8413	53000	0.1143	19.8132	4.3449
0.0355	26.3287	54000	0.1175	19.8088	4.3355
0.0324	26.8164	55000	0.1184	19.7956	4.3260
0.032	27.3038	56000	0.1186	19.7515	4.3229
0.032	27.7915	57000	0.1174	19.7163	4.3173
0.0387	28.2790	58000	0.1175	19.7295	4.3221
0.0345	28.7666	59000	0.1180	19.6634	4.3055
0.0451	29.2541	60000	0.1179	19.6546	4.2992
0.0476	29.7418	61000	0.1178	19.6766	4.2984

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0