w2v2-base-pretrained_lr5e-5_at0.8_da0.3

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 5.8781
• Wer: 0.7877

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
24.822	17.86	250	8.9601	1.0
6.2153	35.71	500	5.9697	1.0
3.5607	53.57	750	5.5112	1.0
3.2305	71.43	1000	5.6378	1.0
2.9631	89.29	1250	5.4385	1.0
2.5463	107.14	1500	5.2806	0.9953
2.1207	125.0	1750	5.3065	0.9492
1.7629	142.86	2000	5.4365	0.9158
1.4631	160.71	2250	5.1678	0.8667
1.2158	178.57	2500	5.3097	0.8428
1.0507	196.43	2750	5.6917	0.8279
0.9326	214.29	3000	5.7407	0.8197
0.8245	232.14	3250	5.5588	0.8039
0.7415	250.0	3500	5.7107	0.7860
0.694	267.86	3750	5.8551	0.7971
0.6634	285.71	4000	5.8781	0.7877

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.0
• Datasets 2.14.6
• Tokenizers 0.14.1