xls-r-1b-bigcgen-female-5hrs-model

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the BIGCGEN - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.7473
• Wer: 0.7080

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30.0

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.4237	100	3.7316	1.0
No log	0.8475	200	2.8335	1.0
No log	1.2712	300	2.0310	1.0
No log	1.6949	400	1.1695	0.9975
5.5638	2.1186	500	1.1579	0.9917
5.5638	2.5424	600	1.1173	0.9931
5.5638	2.9661	700	0.9185	0.7612
5.5638	3.3898	800	0.8461	0.8634
5.5638	3.8136	900	0.7551	0.7391
1.2967	4.2373	1000	0.8461	0.8179
1.2967	4.6610	1100	1.0126	0.9311
1.2967	5.0847	1200	0.7473	0.7077
1.2967	5.5085	1300	0.7741	0.7567
1.2967	5.9322	1400	0.8527	0.7521
0.8911	6.3559	1500	0.9529	0.8052
0.8911	6.7797	1600	1.0235	0.8309

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0