Visualize in Weights & Biases

mHuBERT-147upper-sorbian

This model is a fine-tuned version of utter-project/mHuBERT-147 on the common_voice_17_0 dataset. It achieves the following results on the evaluation set:

  • Loss: 3.2170
  • Wer: 1.0
  • Cer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 100
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
7.2839 3.9216 100 7.4925 1.0 1.0
3.4632 7.8431 200 3.4671 1.0 1.0
3.181 11.7647 300 3.2306 1.0 1.0
3.2284 15.6863 400 3.2231 1.0 1.0
3.2113 19.6078 500 3.2243 1.0 1.0
3.1844 23.5294 600 3.2183 1.0 1.0
3.2644 27.4510 700 3.2180 1.0 1.0
3.2111 31.3725 800 3.2191 1.0 1.0
3.187 35.2941 900 3.2189 1.0 1.0
3.2133 39.2157 1000 3.2203 1.0 1.0
3.2406 43.1373 1100 3.2181 1.0 1.0
3.1993 47.0588 1200 3.2178 1.0 1.0
3.2036 50.9804 1300 3.2169 1.0 1.0
3.2283 54.9020 1400 3.2171 1.0 1.0
3.1854 58.8235 1500 3.2198 1.0 1.0
3.184 62.7451 1600 3.2182 1.0 1.0
3.2253 66.6667 1700 3.2194 1.0 1.0
3.1943 70.5882 1800 3.2194 1.0 1.0
3.201 74.5098 1900 3.2167 1.0 1.0
3.2178 78.4314 2000 3.2180 1.0 1.0
3.2252 82.3529 2100 3.2172 1.0 1.0
3.2081 86.2745 2200 3.2170 1.0 1.0
3.2125 90.1961 2300 3.2170 1.0 1.0
3.23 94.1176 2400 3.2170 1.0 1.0
3.1851 98.0392 2500 3.2170 1.0 1.0

Framework versions

  • Transformers 4.42.0.dev0
  • Pytorch 2.3.1+cu121
  • Datasets 2.19.2
  • Tokenizers 0.19.1
Downloads last month
9
Safetensors
Model size
94.4M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for badrex/mHuBERT-147upper-sorbian

Finetuned
(7)
this model

Evaluation results