eolang/whisper-small-sw

Whisper Small SW-eolang

This model is a fine-tuned version of openai/whisper-small on the Common Voice 17 dataset. It achieves the following results on the evaluation set:

• Loss: 0.5136
• Wer Ortho: 36.8520
• Wer: 27.9511

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_warmup_steps: 50
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ortho	Wer
0.4894	0.1721	500	0.7495	47.1590	39.6183
0.4068	0.3441	1000	0.6356	44.4535	36.3763
0.4137	0.5162	1500	0.5934	41.9094	33.4866
0.3759	0.6882	2000	0.5590	41.4031	33.1765
0.38	0.8603	2500	0.5293	37.2958	28.8699
0.2027	1.0323	3000	0.5235	37.4755	29.0340
0.2089	1.2044	3500	0.5149	35.8239	27.4845
0.2282	1.3765	4000	0.5136	36.8520	27.9511

Framework versions

• Transformers 4.44.2
• Pytorch 2.1.0
• Datasets 2.21.0
• Tokenizers 0.19.1