mms-1b-all-sw-CV_Fleurs_AMMI_ALFFA-5hrs-v1

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Wer: 0.2369
• Cer: 0.0801

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Val Wer	Val Cer
9.5784	1.0	183	0.9998	0.9646
1.9211	2.0	366	0.2887	0.0983
0.6977	3.0	549	0.2641	0.0915
0.6391	4.0	732	0.2568	0.0893
0.59	5.0	915	0.2502	0.0874
0.5654	6.0	1098	0.2489	0.0865
0.5544	7.0	1281	0.2459	0.0853
0.5385	8.0	1464	0.2443	0.0846
0.5256	9.0	1647	0.2442	0.0841
0.5152	10.0	1830	0.2420	0.0837
0.5116	11.0	2013	0.2411	0.0833
0.497	12.0	2196	0.2414	0.0829
0.4969	13.0	2379	0.2400	0.0826
0.4887	14.0	2562	0.2396	0.0824
0.4849	15.0	2745	0.2394	0.0820
0.473	16.0	2928	0.2396	0.0820
0.4732	17.0	3111	0.2403	0.0819
0.4648	18.0	3294	0.2396	0.0817
0.4608	19.0	3477	0.2397	0.0817
0.4583	20.0	3660	0.2397	0.0815
0.4579	21.0	3843	0.2402	0.0817
0.447	22.0	4026	0.2379	0.0813
0.4442	23.0	4209	0.2373	0.0809
0.439	24.0	4392	0.2373	0.0809
0.4381	25.0	4575	0.2376	0.0809
0.4386	26.0	4758	0.2377	0.0807
0.423	27.0	4941	0.2360	0.0804
0.428	28.0	5124	0.2366	0.0803
0.425	29.0	5307	0.2368	0.0804
0.4169	30.0	5490	0.2376	0.0803
0.4253	31.0	5673	0.2367	0.0802
0.4173	32.0	5856	0.2371	0.0802
0.414	33.0	6039	0.2362	0.0800
0.414	34.0	6222	0.2367	0.0803
0.4064	35.0	6405	0.2373	0.0801
0.402	36.0	6588	0.2370	0.0802
0.4035	37.0	6771	0.2369	0.0801

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0