Yasinjan99/xls-r-uyghur-cv13 · Finetune with ug cv13 but keeping get poor wer

Hello, thanks for share the model and training info. I also tried to finetine the xls-r-300m with the ug cv13 with the Fine_Tune_XLS_R_on_Common_Voice tuturial. However, the wer always above 1. My vocab num is 75 and the train and validation loss like bellow:
Step Training Loss Validation Loss Wer
400 3.744300 3.483005 1.000000
800 3.415000 3.326629 1.000000
1200 3.336800 3.522269 1.000000
1600 3.293500 3.235759 1.000000
2000 3.226600 3.225312 1.000000
2400 3.102400 2.962613 1.020783
2800 2.925000 2.913558 1.035719
3200 2.869900 2.885258 1.054565
3600 2.849700 2.889838 1.036600

Here is the model prediction after 20 epoch

Could you give me some advice about the training process? Can not figure out what is wrong here.
Thank you