asr-africa/mms-1B_all_BIG_C_Bemba_5hr_v1

facebook/mms-1b-all

This model is a fine-tuned version of facebook/mms-1b-all on the BIG_C dataset. It achieves the following results on the evaluation set:

• Loss: 0.5796
• Model Preparation Time: 0.0177
• Wer: 0.4635
• Cer: 0.1168

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
5.2767	1.0	154	0.6576	0.0177	0.5943	0.1359
0.7588	2.0	308	0.6085	0.0177	0.5476	0.1268
0.7192	3.0	462	0.5971	0.0177	0.5313	0.1248
0.7048	4.0	616	0.5944	0.0177	0.5432	0.1246
0.6911	5.0	770	0.5889	0.0177	0.5271	0.1226
0.679	6.0	924	0.5702	0.0177	0.5111	0.1205
0.6623	7.0	1078	0.5724	0.0177	0.5046	0.1201
0.65	8.0	1232	0.5605	0.0177	0.5054	0.1209
0.6421	9.0	1386	0.5515	0.0177	0.4993	0.1178
0.626	10.0	1540	0.5518	0.0177	0.4914	0.1166
0.5999	11.0	1694	0.5358	0.0177	0.4910	0.1161
0.5824	12.0	1848	0.5425	0.0177	0.5016	0.1226
0.5723	13.0	2002	0.5325	0.0177	0.5071	0.1195
0.5576	14.0	2156	0.5437	0.0177	0.4880	0.1160
0.5498	15.0	2310	0.5725	0.0177	0.5341	0.1466
0.5383	16.0	2464	0.5814	0.0177	0.4721	0.1141
0.5283	17.0	2618	0.5483	0.0177	0.4819	0.1170
0.5145	18.0	2772	0.5297	0.0177	0.4931	0.1237
0.4977	19.0	2926	0.5283	0.0177	0.4889	0.1208
0.4913	20.0	3080	0.5365	0.0177	0.4776	0.1236
0.4822	21.0	3234	0.5562	0.0177	0.4708	0.1149
0.4768	22.0	3388	0.5493	0.0177	0.4804	0.1160
0.4598	23.0	3542	0.5574	0.0177	0.4736	0.1165
0.4558	24.0	3696	0.5340	0.0177	0.4766	0.1195
0.4516	25.0	3850	0.5703	0.0177	0.4787	0.1143
0.44	26.0	4004	0.5329	0.0177	0.4662	0.1144
0.4322	27.0	4158	0.5790	0.0177	0.4738	0.1136
0.4264	28.0	4312	0.5581	0.0177	0.4729	0.1133
0.4193	29.0	4466	0.5655	0.0177	0.4642	0.1144
0.4115	30.0	4620	0.5521	0.0177	0.4657	0.1173
0.4046	31.0	4774	0.5370	0.0177	0.4626	0.1139
0.4037	32.0	4928	0.5517	0.0177	0.4753	0.1173
0.4016	33.0	5082	0.5733	0.0177	0.4566	0.1119
0.3928	34.0	5236	0.5542	0.0177	0.4715	0.1164
0.3827	35.0	5390	0.5504	0.0177	0.4587	0.1132
0.3828	36.0	5544	0.5541	0.0177	0.4587	0.1126
0.3788	37.0	5698	0.5548	0.0177	0.4551	0.1121
0.371	38.0	5852	0.5574	0.0177	0.4543	0.1131
0.3712	39.0	6006	0.5709	0.0177	0.4600	0.1114
0.3631	40.0	6160	0.5783	0.0177	0.4655	0.1174
0.3561	41.0	6314	0.5753	0.0177	0.4628	0.1151
0.3518	42.0	6468	0.5695	0.0177	0.4691	0.1188
0.3472	43.0	6622	0.5802	0.0177	0.4619	0.1119
0.3423	44.0	6776	0.5796	0.0177	0.4636	0.1146
0.3352	45.0	6930	0.5940	0.0177	0.4585	0.1145
0.3335	46.0	7084	0.5915	0.0177	0.4679	0.1189
0.3294	47.0	7238	0.5885	0.0177	0.4664	0.1165

Framework versions

• Transformers 4.47.0.dev0
• Pytorch 2.1.0+cu118
• Datasets 3.1.0
• Tokenizers 0.20.1