asr-africa/mms-1B_all_BIG_C_Bemba_1hr_v1

facebook/mms-1b-all

This model is a fine-tuned version of facebook/mms-1b-all on the BIG_C dataset. It achieves the following results on the evaluation set:

• Loss: 0.4631
• Model Preparation Time: 0.0111
• Wer: 0.5000
• Cer: 0.0994

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
15.2119	0.9756	15	14.3754	0.0111	1.0	2.8114
13.7154	1.9756	30	11.9551	0.0111	1.0	1.3068
10.7174	2.9756	45	8.7638	0.0111	1.0	0.8494
8.1163	3.9756	60	7.4602	0.0111	1.0	0.9419
7.3697	4.9756	75	7.0681	0.0111	1.0	0.9794
6.7473	5.9756	90	6.4844	0.0111	1.0	0.9688
4.9931	6.9756	105	3.0213	0.0111	0.9998	0.9275
2.4759	7.9756	120	1.4651	0.0111	0.9627	0.3229
1.1333	8.9756	135	0.8066	0.0111	0.6741	0.1628
0.857	9.9756	150	0.7021	0.0111	0.6387	0.1507
0.7852	10.9756	165	0.6550	0.0111	0.6045	0.1409
0.7432	11.9756	180	0.6396	0.0111	0.5860	0.1358
0.7508	12.9756	195	0.6367	0.0111	0.5822	0.1336
0.7122	13.9756	210	0.6270	0.0111	0.5778	0.1333
0.7009	14.9756	225	0.6258	0.0111	0.5638	0.1301
0.699	15.9756	240	0.6172	0.0111	0.5699	0.1318
0.688	16.9756	255	0.6161	0.0111	0.5710	0.1315
0.6861	17.9756	270	0.6167	0.0111	0.5727	0.1319
0.675	18.9756	285	0.6138	0.0111	0.5631	0.1290
0.6587	19.9756	300	0.6150	0.0111	0.5619	0.1295
0.6416	20.9756	315	0.6127	0.0111	0.5574	0.1289
0.6236	21.9756	330	0.6158	0.0111	0.5612	0.1296
0.6339	22.9756	345	0.6235	0.0111	0.5621	0.1316
0.6192	23.9756	360	0.6208	0.0111	0.5574	0.1305
0.5872	24.9756	375	0.6286	0.0111	0.5604	0.1316
0.5714	25.9756	390	0.6294	0.0111	0.5797	0.1374
0.5822	26.9756	405	0.6306	0.0111	0.5786	0.1368
0.5654	27.9756	420	0.6298	0.0111	0.5565	0.1296
0.559	28.9756	435	0.6280	0.0111	0.5597	0.1299
0.5443	29.9756	450	0.6313	0.0111	0.5576	0.1296
0.5395	30.9756	465	0.6402	0.0111	0.5523	0.1277
0.5422	31.9756	480	0.6380	0.0111	0.5445	0.1270
0.5318	32.9756	495	0.6383	0.0111	0.5440	0.1262
0.5233	33.9756	510	0.6411	0.0111	0.5411	0.1266
0.5202	34.9756	525	0.6405	0.0111	0.5432	0.1271
0.5093	35.9756	540	0.6513	0.0111	0.5468	0.1271
0.501	36.9756	555	0.6505	0.0111	0.5461	0.1277
0.5009	37.9756	570	0.6491	0.0111	0.5472	0.1277
0.4849	38.9756	585	0.6738	0.0111	0.5334	0.1239
0.4823	39.9756	600	0.6644	0.0111	0.5387	0.1260
0.4826	40.9756	615	0.6685	0.0111	0.5364	0.1257
0.484	41.9756	630	0.6725	0.0111	0.5360	0.1249
0.4787	42.9756	645	0.6603	0.0111	0.5417	0.1259
0.4711	43.9756	660	0.6696	0.0111	0.5638	0.1327
0.4722	44.9756	675	0.6652	0.0111	0.5385	0.1248
0.4496	45.9756	690	0.6620	0.0111	0.5404	0.1266
0.4453	46.9756	705	0.6950	0.0111	0.5321	0.1248
0.4495	47.9756	720	0.7069	0.0111	0.5290	0.1235
0.4436	48.9756	735	0.6966	0.0111	0.5328	0.1244
0.438	49.9756	750	0.6767	0.0111	0.5351	0.1277
0.4334	50.9756	765	0.6908	0.0111	0.5451	0.1282
0.4227	51.9756	780	0.7179	0.0111	0.5383	0.1260
0.4226	52.9756	795	0.7106	0.0111	0.5377	0.1254
0.4182	53.9756	810	0.7017	0.0111	0.5495	0.1306
0.4153	54.9756	825	0.6970	0.0111	0.5413	0.1295
0.4071	55.9756	840	0.7151	0.0111	0.5394	0.1269
0.4065	56.9756	855	0.7126	0.0111	0.5394	0.1264
0.4015	57.9756	870	0.7199	0.0111	0.5445	0.1279

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.1