asr-africa/mms-1B_all_NCHLT_XHOSA_10hr_v1

facebook mms-1b-all xhosa - Beijuka Bruno

This model is a fine-tuned version of facebook/mms-1b-all on the NCHLT_speech_corpus/Xhosa dataset. It achieves the following results on the evaluation set:

• Loss: 0.3580
• Model Preparation Time: 0.0192
• Wer: 0.5972
• Cer: 0.1361

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
14.2293	0.9976	316	0.2772	0.0192	0.3588	0.0640
1.2546	1.9976	632	0.2685	0.0192	0.3497	0.0616
1.1714	2.9976	948	0.2550	0.0192	0.3326	0.0598
1.1021	3.9976	1264	0.2483	0.0192	0.3212	0.0583
1.0443	4.9976	1580	0.2418	0.0192	0.3073	0.0567
0.9824	5.9976	1896	0.2426	0.0192	0.3003	0.0558
0.959	6.9976	2212	0.2345	0.0192	0.2895	0.0547
0.9257	7.9976	2528	0.2365	0.0192	0.2988	0.0551
0.8916	8.9976	2844	0.2294	0.0192	0.2862	0.0536
0.8608	9.9976	3160	0.2263	0.0192	0.2864	0.0537
0.8313	10.9976	3476	0.2297	0.0192	0.2817	0.0532
0.8067	11.9976	3792	0.2269	0.0192	0.2796	0.0524
0.7644	12.9976	4108	0.2264	0.0192	0.2703	0.0513
0.7608	13.9976	4424	0.2264	0.0192	0.2737	0.0515
0.7311	14.9976	4740	0.2293	0.0192	0.2703	0.0520
0.7109	15.9976	5056	0.2236	0.0192	0.2712	0.0505
0.7245	16.9976	5372	0.2333	0.0192	0.2760	0.0516
0.6952	17.9976	5688	0.2256	0.0192	0.2623	0.0504
0.6798	18.9976	6004	0.2245	0.0192	0.2650	0.0497
0.6525	19.9976	6320	0.2254	0.0192	0.2669	0.0499
0.6366	20.9976	6636	0.2208	0.0192	0.2627	0.0495
0.6263	21.9976	6952	0.2197	0.0192	0.2600	0.0496
0.618	22.9976	7268	0.2227	0.0192	0.2562	0.0491
0.5989	23.9976	7584	0.2228	0.0192	0.2566	0.0496
0.5869	24.9976	7900	0.2278	0.0192	0.2621	0.0496
0.5667	25.9976	8216	0.2260	0.0192	0.2591	0.0498
0.5607	26.9976	8532	0.2242	0.0192	0.2642	0.0503
0.5563	27.9976	8848	0.2235	0.0192	0.2528	0.0488
0.537	28.9976	9164	0.2354	0.0192	0.2699	0.0513
0.53	29.9976	9480	0.2233	0.0192	0.2541	0.0495
0.5143	30.9976	9796	0.2289	0.0192	0.2524	0.0494
0.5118	31.9976	10112	0.2295	0.0192	0.2596	0.0497
0.5041	32.9976	10428	0.2287	0.0192	0.2547	0.0485
0.4918	33.9976	10744	0.2361	0.0192	0.2536	0.0493
0.4825	34.9976	11060	0.2337	0.0192	0.2562	0.0495
0.4758	35.9976	11376	0.2297	0.0192	0.2496	0.0486
0.4525	36.9976	11692	0.2336	0.0192	0.2501	0.0489
0.4564	37.9976	12008	0.2366	0.0192	0.2602	0.0493
0.4555	38.9976	12324	0.2329	0.0192	0.2515	0.0485
0.4471	39.9976	12640	0.2349	0.0192	0.2505	0.0478
0.4324	40.9976	12956	0.2360	0.0192	0.2526	0.0490
0.4219	41.9976	13272	0.2423	0.0192	0.2541	0.0496
0.4172	42.9976	13588	0.2319	0.0192	0.2448	0.0482
0.3976	43.9976	13904	0.2363	0.0192	0.2460	0.0482
0.3991	44.9976	14220	0.2351	0.0192	0.2663	0.0502
0.4055	45.9976	14536	0.2401	0.0192	0.2507	0.0487
0.3889	46.9976	14852	0.2418	0.0192	0.2496	0.0479
0.3931	47.9976	15168	0.2441	0.0192	0.2475	0.0480
0.3651	48.9976	15484	0.2439	0.0192	0.2549	0.0491
0.3707	49.9976	15800	0.2428	0.0192	0.2511	0.0486
0.378	50.9976	16116	0.2371	0.0192	0.2450	0.0479
0.3604	51.9976	16432	0.2462	0.0192	0.2486	0.0483
0.3686	52.9976	16748	0.2457	0.0192	0.2501	0.0484

Framework versions

• Transformers 4.47.1
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0