asr-africa/mms-1B_all_NCHLT_ZULU_1hr_v1

facebook mms-1b-all zulu - Beijuka Bruno

This model is a fine-tuned version of facebook/mms-1b-all on the NCHLT_speech_corpus/Zulu dataset. It achieves the following results on the evaluation set:

• Loss: 0.2337
• Model Preparation Time: 0.0185
• Wer: 0.3740
• Cer: 0.0682

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
134.9856	0.9796	30	13.9196	0.0185	1.0	1.1928
74.6875	1.9796	60	7.4763	0.0185	1.0	0.9660
48.924	2.9796	90	5.0082	0.0185	1.0	0.9261
34.5938	3.9796	120	3.8247	0.0185	1.0	0.9220
28.0008	4.9796	150	3.2622	0.0185	1.0	0.9262
25.6264	5.9796	180	3.0235	0.0185	0.9996	0.9055
23.2532	6.9796	210	2.7683	0.0185	0.9797	0.7992
12.3233	7.9796	240	0.2848	0.0185	0.3328	0.0541
2.5333	8.9796	270	0.1889	0.0185	0.2788	0.0421
2.1275	9.9796	300	0.1637	0.0185	0.2469	0.0373
1.9296	10.9796	330	0.1574	0.0185	0.2447	0.0368
1.7347	11.9796	360	0.1516	0.0185	0.2398	0.0364
1.7104	12.9796	390	0.1484	0.0185	0.2364	0.0355
1.6018	13.9796	420	0.1426	0.0185	0.2300	0.0344
1.5881	14.9796	450	0.1413	0.0185	0.2285	0.0337
1.5097	15.9796	480	0.1381	0.0185	0.2289	0.0337
1.4149	16.9796	510	0.1313	0.0185	0.2180	0.0321
1.4524	17.9796	540	0.1327	0.0185	0.2128	0.0313
1.3885	18.9796	570	0.1311	0.0185	0.2146	0.0323
1.3654	19.9796	600	0.1297	0.0185	0.2101	0.0312
1.2783	20.9796	630	0.1337	0.0185	0.2135	0.0313
1.2367	21.9796	660	0.1309	0.0185	0.2086	0.0310
1.219	22.9796	690	0.1282	0.0185	0.2124	0.0311
1.1463	23.9796	720	0.1277	0.0185	0.2075	0.0307
1.1602	24.9796	750	0.1281	0.0185	0.2101	0.0310
1.1487	25.9796	780	0.1256	0.0185	0.2019	0.0297
1.1645	26.9796	810	0.1259	0.0185	0.1985	0.0296
1.0974	27.9796	840	0.1268	0.0185	0.2023	0.0294
1.0918	28.9796	870	0.1356	0.0185	0.2079	0.0311
1.0377	29.9796	900	0.1270	0.0185	0.2	0.0300
0.9772	30.9796	930	0.1260	0.0185	0.1944	0.0289
0.9718	31.9796	960	0.1258	0.0185	0.1940	0.0289
0.9485	32.9796	990	0.1258	0.0185	0.1962	0.0289
0.9636	33.9796	1020	0.1256	0.0185	0.1910	0.0283
0.9122	34.9796	1050	0.1277	0.0185	0.1932	0.0289
0.9422	35.9796	1080	0.1263	0.0185	0.1906	0.0285
0.9554	36.9796	1110	0.1326	0.0185	0.1940	0.0292
0.9435	37.9796	1140	0.1301	0.0185	0.1925	0.0287
0.8582	38.9796	1170	0.1279	0.0185	0.1895	0.0281
0.8792	39.9796	1200	0.1321	0.0185	0.1944	0.0299
0.8537	40.9796	1230	0.1294	0.0185	0.1932	0.0287
0.8037	41.9796	1260	0.1324	0.0185	0.1936	0.0289
0.7627	42.9796	1290	0.1347	0.0185	0.1992	0.0295
0.836	43.9796	1320	0.1356	0.0185	0.1996	0.0292
0.8203	44.9796	1350	0.1348	0.0185	0.1951	0.0282
0.8289	45.9796	1380	0.1340	0.0185	0.1974	0.0289
0.794	46.9796	1410	0.1305	0.0185	0.1902	0.0280
0.7716	47.9796	1440	0.1307	0.0185	0.1902	0.0276
0.7461	48.9796	1470	0.1327	0.0185	0.1925	0.0279

Framework versions

• Transformers 4.47.1
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0