README.md

metadata

tags:
  - generated_from_trainer
model-index:
  - name: wavlm-large-timit-punctuation
    results: []

wavlm-large-timit-punctuation

This model is a fine-tuned version of microsoft/wavlm-large on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3360
• Wer: 0.2580

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.2206	1.0	500	3.1111	1.0
2.4555	2.01	1000	1.0331	0.7992
0.9277	3.01	1500	0.5219	0.4888
0.5215	4.02	2000	0.3833	0.3981
0.3557	5.02	2500	0.3330	0.3570
0.2715	6.02	3000	0.3084	0.3255
0.2139	7.03	3500	0.2969	0.3129
0.1858	8.03	4000	0.2884	0.3029
0.1563	9.04	4500	0.2860	0.2960
0.149	10.04	5000	0.2972	0.2918
0.1343	11.04	5500	0.3161	0.2927
0.11	12.05	6000	0.3061	0.2788
0.0982	13.05	6500	0.2983	0.2802
0.0967	14.06	7000	0.3280	0.2768
0.0873	15.06	7500	0.3185	0.2721
0.0809	16.06	8000	0.3121	0.2694
0.0787	17.07	8500	0.3177	0.2643
0.0709	18.07	9000	0.3189	0.2657
0.0712	19.08	9500	0.3213	0.2628
0.0621	20.08	10000	0.3206	0.2600
0.0601	21.08	10500	0.3191	0.2600
0.0605	22.09	11000	0.3241	0.2591
0.058	23.09	11500	0.3230	0.2584
0.0503	24.1	12000	0.3346	0.2602
0.0498	25.1	12500	0.3359	0.2593
0.0506	26.1	13000	0.3339	0.2592
0.0468	27.11	13500	0.3357	0.2563
0.0422	28.11	14000	0.3368	0.2568
0.0512	29.12	14500	0.3360	0.2580

Framework versions

• Transformers 4.19.2
• Pytorch 1.8.2+cu111
• Datasets 1.17.0
• Tokenizers 0.11.6