checkpoints

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2593
• Wer: 0.3195

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.9162	0.1621	100	0.5752	0.4898
0.4731	0.3241	200	0.3338	0.3774
0.4288	0.4862	300	0.3089	0.3573
0.3767	0.6483	400	0.3064	0.3607
0.4865	0.8104	500	0.3002	0.3558
0.3979	0.9724	600	0.2988	0.3565
0.3744	1.1345	700	0.2918	0.3596
0.4063	1.2966	800	0.2944	0.3497
0.3615	1.4587	900	0.2875	0.3543
0.3965	1.6207	1000	0.2790	0.3369
0.3846	1.7828	1100	0.2788	0.3372
0.3838	1.9449	1200	0.2747	0.3297
0.4338	2.1070	1300	0.2698	0.3361
0.2994	2.2690	1400	0.2688	0.3263
0.3604	2.4311	1500	0.2718	0.3259
0.3553	2.5932	1600	0.2687	0.3289
0.3616	2.7553	1700	0.2674	0.3232
0.3265	2.9173	1800	0.2656	0.3119
0.2892	3.0794	1900	0.2644	0.3221
0.2685	3.2415	2000	0.2646	0.3153
0.3268	3.4036	2100	0.2640	0.3183
0.2855	3.5656	2200	0.2620	0.3202
0.3351	3.7277	2300	0.2604	0.3195
0.4487	3.8898	2400	0.2593	0.3195

Framework versions

• Transformers 4.49.0.dev0
• Pytorch 2.6.0+cu126
• Datasets 3.2.0
• Tokenizers 0.21.0