asr-africa/mms-1B_all_NCHLT_XHOSA_1hr_v1

facebook mms-1b-all xhosa - Beijuka Bruno

This model is a fine-tuned version of facebook/mms-1b-all on the NCHLT_speech_corpus/Xhosa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2280
• Model Preparation Time: 0.0199
• Wer: 0.3297
• Cer: 0.0622

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
161.2211	0.9888	33	16.9451	0.0199	4.5936	1.5848
67.6701	1.9888	66	3.0671	0.0199	1.0	0.8589
14.5117	2.9888	99	0.4740	0.0199	0.5122	0.0912
4.1033	3.9888	132	0.2786	0.0199	0.3982	0.0621
3.1614	4.9888	165	0.2421	0.0199	0.3497	0.0555
2.9473	5.9888	198	0.2270	0.0199	0.3260	0.0517
2.7283	6.9888	231	0.2164	0.0199	0.3242	0.05
2.5382	7.9888	264	0.2095	0.0199	0.3012	0.0475
2.4532	8.9888	297	0.2051	0.0199	0.3016	0.0479
2.3352	9.9888	330	0.1977	0.0199	0.3037	0.0465
2.2913	10.9888	363	0.1966	0.0199	0.2906	0.0460
2.2131	11.9888	396	0.1998	0.0199	0.3101	0.0464
2.1296	12.9888	429	0.1912	0.0199	0.2821	0.0444
2.0863	13.9888	462	0.1934	0.0199	0.2796	0.0442
2.016	14.9888	495	0.1927	0.0199	0.2761	0.0439
1.9625	15.9888	528	0.1896	0.0199	0.2758	0.0438
1.9719	16.9888	561	0.1921	0.0199	0.2581	0.0422
1.8811	17.9888	594	0.1910	0.0199	0.2736	0.0435
1.759	18.9888	627	0.1913	0.0199	0.2680	0.0433
1.7474	19.9888	660	0.1883	0.0199	0.2602	0.0426
1.6931	20.9888	693	0.1925	0.0199	0.2669	0.0430
1.6515	21.9888	726	0.1879	0.0199	0.2591	0.0423
1.6038	22.9888	759	0.1919	0.0199	0.2676	0.0431
1.608	23.9888	792	0.1960	0.0199	0.2665	0.0426
1.6418	24.9888	825	0.1940	0.0199	0.2612	0.0418
1.5068	25.9888	858	0.1985	0.0199	0.2609	0.0427
1.5171	26.9888	891	0.1932	0.0199	0.2612	0.0423

Framework versions

• Transformers 4.47.1
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0