asr-africa/mms-1B_all_DigitalUmuganda_Afrivoice_Shona_5hr_v1

facebook/mms-1b-all

This model is a fine-tuned version of facebook/mms-1b-all on the DigitalUmuganda dataset. It achieves the following results on the evaluation set:

• Loss: 0.2197
• Model Preparation Time: 0.011
• Wer: 0.2845
• Cer: 0.0508

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 150
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
57.4573	0.9818	27	11.9211	0.011	1.8495	2.2226
32.7495	1.9727	54	3.8209	0.011	1.0	0.9976
11.1389	2.9636	81	1.5552	0.011	0.9324	0.3299
2.4363	3.9909	109	0.2450	0.011	0.3251	0.0558
1.0951	4.9818	136	0.2150	0.011	0.3210	0.0535
1.0093	5.9727	163	0.2058	0.011	0.3108	0.0515
0.9759	6.9636	190	0.2019	0.011	0.3125	0.0516
0.9196	7.9909	218	0.1983	0.011	0.3074	0.0509
0.9158	8.9818	245	0.1957	0.011	0.3118	0.0509
0.9027	9.9727	272	0.1943	0.011	0.3142	0.0506
0.8848	10.9636	299	0.1929	0.011	0.3042	0.0498
0.8314	11.9909	327	0.1939	0.011	0.2977	0.0491
0.85	12.9818	354	0.1906	0.011	0.3033	0.0496
0.8357	13.9727	381	0.1914	0.011	0.3013	0.0486
0.8153	14.9636	408	0.1901	0.011	0.2972	0.0482
0.7754	15.9909	436	0.1918	0.011	0.2996	0.0489
0.7957	16.9818	463	0.1894	0.011	0.3071	0.0490
0.7801	17.9727	490	0.1887	0.011	0.2955	0.0478
0.7703	18.9636	517	0.1904	0.011	0.3008	0.0481
0.7231	19.9909	545	0.1904	0.011	0.2994	0.0486
0.7414	20.9818	572	0.1883	0.011	0.2974	0.0480
0.7336	21.9727	599	0.1885	0.011	0.2889	0.0474
0.7154	22.9636	626	0.1887	0.011	0.2909	0.0473
0.6867	23.9909	654	0.1876	0.011	0.2904	0.0473
0.7064	24.9818	681	0.1880	0.011	0.2916	0.0474
0.6896	25.9727	708	0.1894	0.011	0.2943	0.0473
0.68	26.9636	735	0.1896	0.011	0.3057	0.0476
0.6438	27.9909	763	0.1893	0.011	0.3011	0.0478
0.658	28.9818	790	0.1901	0.011	0.2940	0.0472
0.6452	29.9727	817	0.1910	0.011	0.2962	0.0476
0.6396	30.9636	844	0.1914	0.011	0.2896	0.0473
0.6067	31.9909	872	0.1941	0.011	0.2894	0.0473

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.0.2
• Tokenizers 0.20.1