mms-1b-bigcgen-female-30hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.5369

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.2597	0.0501	100	inf	1.0025
6.2197	0.1002	200	inf	0.9946
5.2752	0.1503	300	inf	1.0316
2.3626	0.2004	400	inf	0.5998
1.7448	0.2504	500	inf	0.5814
1.6955	0.3005	600	inf	0.5709
1.6841	0.3506	700	inf	0.5596
1.693	0.4007	800	inf	0.5639
1.688	0.4508	900	inf	0.5555
1.5718	0.5009	1000	inf	0.5476
1.5855	0.5510	1100	inf	0.5482
1.4783	0.6011	1200	inf	0.5471
1.5198	0.6511	1300	inf	0.5476
1.4941	0.7012	1400	inf	0.5469
1.5916	0.7513	1500	inf	0.5426
1.4683	0.8014	1600	inf	0.5460
1.486	0.8515	1700	inf	0.5528
1.4353	0.9016	1800	inf	0.5435
1.6166	0.9517	1900	inf	0.5541
1.531	1.0015	2000	inf	0.5535
1.5441	1.0516	2100	inf	0.5478
1.3459	1.1017	2200	inf	0.5274
1.357	1.1518	2300	inf	0.5269
1.4464	1.2019	2400	inf	0.5226
1.4326	1.2519	2500	inf	0.5369

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0