mms-1b-bigcgen-female-5hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

• Loss: inf
• Wer: 0.5367

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 2500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.2275	0.3003	100	inf	1.0107
6.1906	0.6006	200	inf	1.0102
5.5599	0.9009	300	inf	1.0173
4.5221	1.2012	400	inf	0.8938
2.0302	1.5015	500	inf	0.6640
2.0609	1.8018	600	inf	0.6136
1.824	2.1021	700	inf	0.5930
1.8659	2.4024	800	inf	0.5936
1.734	2.7027	900	inf	0.6061
1.5581	3.0030	1000	inf	0.5766
1.6672	3.3033	1100	inf	0.5498
1.5631	3.6036	1200	inf	0.5364
1.5806	3.9039	1300	inf	0.5471
1.4877	4.2042	1400	inf	0.5514
1.4917	4.5045	1500	inf	0.5358
1.4686	4.8048	1600	inf	0.5503
1.4303	5.1051	1700	inf	0.5448
1.3914	5.4054	1800	inf	0.5442
1.427	5.7057	1900	inf	0.5621
1.4636	6.0060	2000	inf	0.5768
1.4853	6.3063	2100	inf	0.5226
1.3929	6.6066	2200	inf	0.5373
1.343	6.9069	2300	inf	0.5362
1.4545	7.2072	2400	inf	0.5405
1.3445	7.5075	2500	inf	0.5371

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0