mms-1b-nyagen-female-model

This model is a fine-tuned version of facebook/mms-1b-all on the NYAGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.2166
• Wer: 0.2541

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
7.8679	0.3058	100	0.9128	0.6897
0.627	0.6116	200	0.3265	0.4386
0.5004	0.9174	300	0.2951	0.4040
0.4611	1.2232	400	0.2758	0.3659
0.4455	1.5291	500	0.2667	0.3599
0.4142	1.8349	600	0.2601	0.3514
0.3706	2.1407	700	0.2521	0.3479
0.3867	2.4465	800	0.2461	0.3128
0.3537	2.7523	900	0.2449	0.3158
0.3821	3.0581	1000	0.2413	0.2932
0.3626	3.3639	1100	0.2360	0.3083
0.3312	3.6697	1200	0.2336	0.2997
0.3322	3.9755	1300	0.2285	0.2967
0.3654	4.2813	1400	0.2235	0.2852
0.3241	4.5872	1500	0.2198	0.2807
0.2908	4.8930	1600	0.2167	0.2767
0.3299	5.1988	1700	0.2170	0.2747
0.3128	5.5046	1800	0.2147	0.2687
0.3094	5.8104	1900	0.2140	0.2742
0.309	6.1162	2000	0.2158	0.2702
0.3075	6.4220	2100	0.2127	0.2652
0.2823	6.7278	2200	0.2155	0.2672
0.3062	7.0336	2300	0.2133	0.2556
0.3012	7.3394	2400	0.2166	0.2541

Framework versions

• Transformers 4.48.0.dev0
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0