asr-africa/mms-1B_all_NCHLT_ZULU_50hr_v1

facebook mms-1b-all zulu - Beijuka Bruno

This model is a fine-tuned version of facebook/mms-1b-all on the NCHLT_speech_corpus/Zulu dataset. It achieves the following results on the evaluation set:

• Loss: 0.2443
• Model Preparation Time: 0.0119
• Wer: 0.6532
• Cer: 0.1372

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
2.9828	1.0	2523	0.2050	0.0119	0.3254	0.0439
0.2496	2.0	5046	0.1529	0.0119	0.2548	0.0359
0.2165	3.0	7569	0.1458	0.0119	0.2314	0.0314
0.2017	4.0	10092	0.1427	0.0119	0.2376	0.0324
0.1926	5.0	12615	0.1349	0.0119	0.2220	0.0301
0.1791	6.0	15138	0.1320	0.0119	0.2104	0.0296
0.1704	7.0	17661	0.1294	0.0119	0.2200	0.0303
0.1639	8.0	20184	0.1298	0.0119	0.2159	0.0303
0.1581	9.0	22707	0.1304	0.0119	0.2124	0.0295
0.154	10.0	25230	0.1306	0.0119	0.2046	0.0284
0.1489	11.0	27753	0.1291	0.0119	0.2032	0.0282
0.1454	12.0	30276	0.1301	0.0119	0.2072	0.0298
0.1424	13.0	32799	0.1287	0.0119	0.2088	0.0293
0.138	14.0	35322	0.1303	0.0119	0.1994	0.0279
0.1368	15.0	37845	0.1292	0.0119	0.2267	0.0339
0.1339	16.0	40368	0.1259	0.0119	0.2470	0.0342
0.1319	17.0	42891	0.1294	0.0119	0.2092	0.0291
0.1285	18.0	45414	0.1324	0.0119	0.2023	0.0284
0.125	19.0	47937	0.1307	0.0119	0.2059	0.0287
0.1234	20.0	50460	0.1283	0.0119	0.2048	0.0290
0.1197	21.0	52983	0.1273	0.0119	0.2068	0.0288
0.1196	22.0	55506	0.1306	0.0119	0.2126	0.0293
0.1157	23.0	58029	0.1302	0.0119	0.2050	0.0289
0.1134	24.0	60552	0.1289	0.0119	0.2061	0.0287

Framework versions

• Transformers 4.47.0
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0