wav2vec2-xls-r-1b-faroese-100h-30-epochs_20250126_v2

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1065
• Wer: 18.8527
• Cer: 4.1045

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 5000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.4856	0.4877	1000	0.8786	82.4118	26.5338
0.6959	0.9754	2000	0.4059	49.6850	13.2626
0.4978	1.4628	3000	0.2672	36.8991	9.6228
0.4215	1.9505	4000	0.2359	33.7005	8.5821
0.3261	2.4379	5000	0.2273	31.9910	8.2815
0.3388	2.9256	6000	0.2162	31.0261	7.9927
0.2987	3.4131	7000	0.1988	29.2814	7.4294
0.2824	3.9008	8000	0.1919	28.5192	7.1311
0.257	4.3882	9000	0.1861	27.9861	7.1059
0.2668	4.8759	10000	0.1758	27.2679	6.8108
0.2438	5.3633	11000	0.1699	26.8450	6.6419
0.2218	5.8510	12000	0.1704	26.7040	6.6222
0.194	6.3385	13000	0.1623	25.9638	6.3760
0.1997	6.8261	14000	0.1550	25.5805	6.3216
0.186	7.3136	15000	0.1493	25.5584	6.1243
0.1686	7.8013	16000	0.1597	25.3998	6.2198
0.1433	8.2887	17000	0.1493	24.6552	5.8813
0.1581	8.7764	18000	0.1459	24.7962	5.9847
0.1245	9.2638	19000	0.1425	24.3865	5.8190
0.1396	9.7515	20000	0.1576	24.5407	5.8900
0.1188	10.2390	21000	0.1410	23.7300	5.6407
0.1131	10.7267	22000	0.1373	23.4524	5.5586
0.1267	11.2141	23000	0.1397	23.7741	5.6075
0.1192	11.7018	24000	0.1346	23.4084	5.4482
0.1073	12.1892	25000	0.1325	22.8312	5.3811
0.098	12.6769	26000	0.1227	22.3642	5.1681
0.0939	13.1644	27000	0.1307	22.7695	5.2856
0.0929	13.6520	28000	0.1274	22.2629	5.1854
0.0787	14.1395	29000	0.1265	21.9853	5.0268
0.0805	14.6272	30000	0.1239	22.1131	5.0923
0.0766	15.1146	31000	0.1226	21.7386	4.9905
0.0745	15.6023	32000	0.1189	21.4786	4.9124
0.0663	16.0897	33000	0.1224	21.7650	4.9250
0.0638	16.5774	34000	0.1188	21.0689	4.8185
0.0609	17.0649	35000	0.1177	20.9058	4.7420
0.0677	17.5525	36000	0.1140	21.0116	4.7546
0.0516	18.0400	37000	0.1180	20.8794	4.7025
0.0488	18.5277	38000	0.1177	20.4653	4.6307
0.0512	19.0151	39000	0.1135	20.1789	4.5447
0.0464	19.5028	40000	0.1163	20.2758	4.5013
0.0465	19.9905	41000	0.1125	20.1877	4.4635
0.0536	20.4779	42000	0.1129	20.0247	4.4477
0.0417	20.9656	43000	0.1131	19.7383	4.3830
0.038	21.4531	44000	0.1117	19.8881	4.4027
0.0454	21.9407	45000	0.1105	19.6942	4.3514
0.031	22.4282	46000	0.1130	19.6193	4.3388
0.0373	22.9159	47000	0.1100	19.4563	4.2788
0.0319	23.4033	48000	0.1107	19.4519	4.2717
0.035	23.8910	49000	0.1112	19.2933	4.2299
0.0327	24.3784	50000	0.1111	19.1964	4.1999
0.0261	24.8661	51000	0.1056	19.0906	4.1842
0.0321	25.3536	52000	0.1059	18.9496	4.1447
0.0301	25.8413	53000	0.1058	19.0906	4.1558
0.0281	26.3287	54000	0.1091	18.9893	4.1344
0.0246	26.8164	55000	0.1062	18.8836	4.1100
0.0238	27.3038	56000	0.1064	18.8615	4.1076
0.0305	27.7915	57000	0.1068	18.9188	4.1195
0.0293	28.2790	58000	0.1067	18.8791	4.1060
0.0288	28.7666	59000	0.1066	18.8836	4.1131
0.0345	29.2541	60000	0.1065	18.8439	4.1029
0.0365	29.7418	61000	0.1065	18.8527	4.1045

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0