mms-1B_all_NCHLT_speech_corpus_Fleurs_Xhosa_63hr_v1

This model is a fine-tuned version of facebook/mms-1b-all on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2408
• Wer: 0.3782
• Cer: 0.0618

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
2.6309	0.9997	2908	0.2678	0.4025	0.0688
1.073	1.9997	5816	0.2657	0.3859	0.0670
1.008	2.9997	8724	0.2572	0.3778	0.0653
0.9633	3.9997	11632	0.2529	0.3743	0.0657
0.9271	4.9997	14540	0.2524	0.3735	0.0653
0.9031	5.9997	17448	0.2508	0.3658	0.0642
0.88	6.9997	20356	0.2519	0.3752	0.0652
0.8533	7.9997	23264	0.2493	0.3671	0.0640
0.8345	8.9997	26172	0.2512	0.3634	0.0632
0.8176	9.9997	29080	0.2495	0.3576	0.0627
0.8029	10.9997	31988	0.2506	0.3672	0.0638
0.7827	11.9997	34896	0.2474	0.3582	0.0626
0.7704	12.9997	37804	0.2485	0.3497	0.0619
0.7573	13.9997	40712	0.2456	0.3543	0.0619
0.7443	14.9997	43620	0.2483	0.3541	0.0622
0.733	15.9997	46528	0.2492	0.3621	0.0619
0.7218	16.9997	49436	0.2461	0.3565	0.0610
0.711	17.9997	52344	0.2462	0.3532	0.0619
0.6975	18.9997	55252	0.2484	0.3536	0.0615
0.684	19.9997	58160	0.2466	0.3599	0.0617
0.6802	20.9997	61068	0.2466	0.3486	0.0600
0.668	21.9997	63976	0.2433	0.3527	0.0606
0.6588	22.9997	66884	0.2450	0.3469	0.0609
0.6503	23.9997	69792	0.2425	0.3495	0.0607
0.6474	24.9997	72700	0.2478	0.3495	0.0609
0.6311	25.9997	75608	0.2454	0.3449	0.0598
0.6254	26.9997	78516	0.2497	0.3523	0.0611
0.6187	27.9997	81424	0.2444	0.3466	0.0597
0.6091	28.9997	84332	0.2460	0.3395	0.0592
0.6016	29.9997	87240	0.2458	0.3373	0.0590
0.5943	30.9997	90148	0.2442	0.3377	0.0588
0.5905	31.9997	93056	0.2438	0.3338	0.0582
0.5828	32.9997	95964	0.2510	0.3438	0.0597
0.5758	33.9997	98872	0.2460	0.3383	0.0588
0.5711	34.9997	101780	0.2463	0.3338	0.0580
0.5628	35.9997	104688	0.2459	0.3434	0.0589
0.552	36.9997	107596	0.2476	0.3383	0.0589
0.552	37.9997	110504	0.2498	0.3440	0.0591
0.544	38.9997	113412	0.2469	0.3414	0.0587
0.5354	39.9997	116320	0.2494	0.3377	0.0583
0.5316	40.9997	119228	0.2514	0.3336	0.0582
0.5265	41.9997	122136	0.2539	0.3384	0.0592

Framework versions

• Transformers 4.47.1
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0