Hugging Face's logo

utakumi
/
Hubert-common_voice-ja-demo-roma-cosine-3e-4

Automatic Speech Recognition
Transformers
Safetensors
Japanese
hubert
mozilla-foundation/common_voice_13_0
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
Hubert-common_voice-ja-demo-roma-cosine-3e-4 / README.md
utakumi's picture
utakumi
End of training
396d19a verified
preview code
|
raw
history blame contribute delete
15.7 kB
metadata
library_name: transformers
language:
  - ja
license: apache-2.0
base_model: rinna/japanese-hubert-base
tags:
  - automatic-speech-recognition
  - mozilla-foundation/common_voice_13_0
  - generated_from_trainer
datasets:
  - common_voice_13_0
metrics:
  - wer
model-index:
  - name: Hubert-common_voice-ja-demo-roma-cosine-3e-4
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: MOZILLA-FOUNDATION/COMMON_VOICE_13_0 - JA
          type: common_voice_13_0
          config: ja
          split: test
          args: 'Config: ja, Training split: train+validation, Eval split: test'
        metrics:
          - name: Wer
            type: wer
            value: 0.9987900786448881

Hubert-common_voice-ja-demo-roma-cosine-3e-4

This model is a fine-tuned version of rinna/japanese-hubert-base on the MOZILLA-FOUNDATION/COMMON_VOICE_13_0 - JA dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3032
  • Wer: 0.9988
  • Cer: 0.1667

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 12500
  • num_epochs: 50.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer Cer
No log 0.2660 100 15.8236 1.7925 1.6163
No log 0.5319 200 6.4207 1.0 0.9276
No log 0.7979 300 5.4193 1.0 0.9276
No log 1.0638 400 4.9073 1.0 0.9276
7.456 1.3298 500 4.3586 1.0 0.9276
7.456 1.5957 600 3.8341 1.0 0.9276
7.456 1.8617 700 3.4148 1.0 0.9276
7.456 2.1277 800 3.1047 1.0 0.9276
7.456 2.3936 900 2.9612 1.0 0.9276
3.2563 2.6596 1000 2.9027 1.0 0.9276
3.2563 2.9255 1100 2.8767 1.0 0.9276
3.2563 3.1915 1200 2.8588 1.0 0.9276
3.2563 3.4574 1300 2.8038 1.0 0.9276
3.2563 3.7234 1400 1.9933 1.0 0.8359
2.5032 3.9894 1500 1.1381 0.9998 0.3811
2.5032 4.2553 1600 0.8252 0.9984 0.2746
2.5032 4.5213 1700 0.6763 0.9974 0.2507
2.5032 4.7872 1800 0.6011 0.9980 0.2417
2.5032 5.0532 1900 0.5514 0.9976 0.2308
0.678 5.3191 2000 0.5129 0.9986 0.2249
0.678 5.5851 2100 0.4958 0.9944 0.2325
0.678 5.8511 2200 0.4717 0.9978 0.2195
0.678 6.1170 2300 0.4652 0.9992 0.2173
0.678 6.3830 2400 0.4555 0.9974 0.2122
0.4238 6.6489 2500 0.4361 0.9988 0.2116
0.4238 6.9149 2600 0.4181 0.9990 0.2063
0.4238 7.1809 2700 0.4186 0.9958 0.2012
0.4238 7.4468 2800 0.4254 0.9994 0.2035
0.4238 7.7128 2900 0.4014 0.9984 0.1981
0.3375 7.9787 3000 0.3877 0.9976 0.1980
0.3375 8.2447 3100 0.3868 0.9982 0.1926
0.3375 8.5106 3200 0.3740 0.9978 0.1903
0.3375 8.7766 3300 0.3645 0.9982 0.1855
0.3375 9.0426 3400 0.3586 0.9992 0.1824
0.2553 9.3085 3500 0.3328 0.9980 0.1775
0.2553 9.5745 3600 0.3401 0.9978 0.1744
0.2553 9.8404 3700 0.3124 0.9976 0.1727
0.2553 10.1064 3800 0.3225 0.9988 0.1709
0.2553 10.3723 3900 0.3311 0.9974 0.1760
0.2035 10.6383 4000 0.3098 0.9980 0.1705
0.2035 10.9043 4100 0.3244 0.9980 0.1714
0.2035 11.1702 4200 0.3280 0.9925 0.1686
0.2035 11.4362 4300 0.3134 0.9984 0.1705
0.2035 11.7021 4400 0.3025 0.9988 0.1667
0.1772 11.9681 4500 0.3156 0.9980 0.1690
0.1772 12.2340 4600 0.3213 0.9968 0.1657
0.1772 12.5 4700 0.3184 0.9976 0.1702
0.1772 12.7660 4800 0.3348 0.9990 0.1659
0.1772 13.0319 4900 0.3175 0.9978 0.1655
0.1542 13.2979 5000 0.3414 0.9998 0.1680
0.1542 13.5638 5100 0.3143 0.9994 0.1701
0.1542 13.8298 5200 0.3204 0.9986 0.1688
0.1542 14.0957 5300 0.3549 0.9990 0.1662
0.1542 14.3617 5400 0.4091 0.9974 0.1666
0.1449 14.6277 5500 0.3908 0.9986 0.1676
0.1449 14.8936 5600 0.3706 0.9984 0.1662
0.1449 15.1596 5700 0.3972 0.9972 0.1641
0.1449 15.4255 5800 0.3462 0.9984 0.1653
0.1449 15.6915 5900 0.3544 0.9984 0.1699
0.1396 15.9574 6000 0.3397 0.9988 0.1682
0.1396 16.2234 6100 0.3452 0.9984 0.1680
0.1396 16.4894 6200 0.3534 0.9982 0.1665
0.1396 16.7553 6300 0.3502 0.9986 0.1703
0.1396 17.0213 6400 0.3475 0.9978 0.1701
0.1293 17.2872 6500 0.3350 0.9988 0.1681
0.1293 17.5532 6600 0.3442 0.9978 0.1694
0.1293 17.8191 6700 0.3342 0.9988 0.1687
0.1293 18.0851 6800 0.3669 0.9986 0.1696
0.1293 18.3511 6900 0.3404 0.9970 0.1691
0.1276 18.6170 7000 0.3464 0.9990 0.1679
0.1276 18.8830 7100 0.3496 0.9984 0.1695
0.1276 19.1489 7200 0.3436 0.9968 0.1698
0.1276 19.4149 7300 0.3605 0.9954 0.1690
0.1276 19.6809 7400 0.3582 0.9974 0.1687
0.1264 19.9468 7500 0.3576 0.9982 0.1740
0.1264 20.2128 7600 0.3669 0.9986 0.1726
0.1264 20.4787 7700 0.3618 0.9980 0.1706
0.1264 20.7447 7800 0.3475 0.9990 0.1746
0.1264 21.0106 7900 0.3425 0.9972 0.1715
0.1219 21.2766 8000 0.3685 0.9984 0.1716
0.1219 21.5426 8100 0.3803 0.9990 0.1756
0.1219 21.8085 8200 0.3663 0.9984 0.1797
0.1219 22.0745 8300 0.3642 0.9978 0.1710
0.1219 22.3404 8400 0.3805 0.9988 0.1737
0.1177 22.6064 8500 0.3630 0.9986 0.1747
0.1177 22.8723 8600 0.4001 0.9974 0.1753
0.1177 23.1383 8700 0.3758 0.9978 0.1758
0.1177 23.4043 8800 0.3771 0.9984 0.1747
0.1177 23.6702 8900 0.4001 0.9984 0.1794
0.1241 23.9362 9000 0.3929 0.9998 0.1769
0.1241 24.2021 9100 0.3732 0.9992 0.1752
0.1241 24.4681 9200 0.3813 0.9984 0.1738
0.1241 24.7340 9300 0.4128 0.9990 0.1794
0.1241 25.0 9400 0.3756 0.9990 0.1751
0.121 25.2660 9500 0.3916 0.9990 0.1797
0.121 25.5319 9600 0.3882 0.9984 0.1824
0.121 25.7979 9700 0.3917 0.9976 0.1838
0.121 26.0638 9800 0.3928 0.9984 0.1770
0.121 26.3298 9900 0.3929 0.9996 0.1801
0.1206 26.5957 10000 0.3985 0.9988 0.1781
0.1206 26.8617 10100 0.3799 0.9994 0.1771
0.1206 27.1277 10200 0.4023 0.9994 0.1786
0.1206 27.3936 10300 0.4000 0.9992 0.1784
0.1206 27.6596 10400 0.3756 0.9976 0.1825
0.124 27.9255 10500 0.3971 0.9986 0.1779
0.124 28.1915 10600 0.4240 0.9996 0.1789
0.124 28.4574 10700 0.3718 0.9980 0.1792
0.124 28.7234 10800 0.4114 0.9986 0.1800
0.124 28.9894 10900 0.4174 0.9978 0.1800
0.122 29.2553 11000 0.4062 0.9988 0.1853
0.122 29.5213 11100 0.4203 0.9978 0.1861
0.122 29.7872 11200 0.4376 0.9986 0.1861
0.122 30.0532 11300 0.4094 0.9992 0.1812
0.122 30.3191 11400 0.4100 0.9988 0.1819
0.125 30.5851 11500 0.3997 0.9982 0.1869
0.125 30.8511 11600 0.4437 0.9990 0.1820
0.125 31.1170 11700 0.4423 0.9990 0.1858
0.125 31.3830 11800 0.4217 0.9988 0.1895
0.125 31.6489 11900 0.4612 0.9992 0.1966
0.1294 31.9149 12000 0.4386 0.9974 0.1862
0.1294 32.1809 12100 0.4278 0.9984 0.1892
0.1294 32.4468 12200 0.4187 0.9984 0.1856
0.1294 32.7128 12300 0.4047 0.9986 0.1829
0.1294 32.9787 12400 0.4231 0.9980 0.1852
0.1275 33.2447 12500 0.4124 0.9994 0.1843
0.1275 33.5106 12600 0.4191 0.9994 0.1870
0.1275 33.7766 12700 0.4846 0.9980 0.1927
0.1275 34.0426 12800 0.4212 0.9984 0.1845
0.1275 34.3085 12900 0.4326 0.9984 0.1837
0.134 34.5745 13000 0.4104 0.9992 0.1880
0.134 34.8404 13100 0.3965 0.9978 0.1877
0.134 35.1064 13200 0.4147 0.9994 0.1844
0.134 35.3723 13300 0.4251 0.9978 0.1859
0.134 35.6383 13400 0.4458 0.9994 0.1902
0.1293 35.9043 13500 0.4354 0.9992 0.1942
0.1293 36.1702 13600 0.4198 0.9996 0.1863
0.1293 36.4362 13700 0.4279 0.9986 0.1891
0.1293 36.7021 13800 0.4115 0.9976 0.1846
0.1293 36.9681 13900 0.4359 0.9986 0.1868
0.1193 37.2340 14000 0.4316 0.9992 0.1905
0.1193 37.5 14100 0.4389 0.9994 0.1899
0.1193 37.7660 14200 0.4215 0.9992 0.1825
0.1193 38.0319 14300 0.4793 0.9992 0.1890
0.1193 38.2979 14400 0.4382 0.9984 0.1854
0.1132 38.5638 14500 0.4011 0.9976 0.1824
0.1132 38.8298 14600 0.4283 0.9950 0.1798
0.1132 39.0957 14700 0.4304 0.9980 0.1803
0.1132 39.3617 14800 0.4049 0.9984 0.1811
0.1132 39.6277 14900 0.4146 0.9988 0.1785
0.0949 39.8936 15000 0.4499 0.9998 0.1817
0.0949 40.1596 15100 0.4205 0.9978 0.1791
0.0949 40.4255 15200 0.4419 0.9986 0.1807
0.0949 40.6915 15300 0.4283 0.9982 0.1801
0.0949 40.9574 15400 0.4327 0.9996 0.1769
0.0876 41.2234 15500 0.4488 0.9996 0.1781
0.0876 41.4894 15600 0.4194 0.9990 0.1736
0.0876 41.7553 15700 0.4320 0.9992 0.1754
0.0876 42.0213 15800 0.4347 0.9990 0.1729
0.0876 42.2872 15900 0.4819 0.9994 0.1744
0.0725 42.5532 16000 0.4491 0.9990 0.1752
0.0725 42.8191 16100 0.4537 0.9986 0.1741
0.0725 43.0851 16200 0.4588 0.9984 0.1717
0.0725 43.3511 16300 0.4417 0.9982 0.1715
0.0725 43.6170 16400 0.4554 0.9984 0.1729
0.0615 43.8830 16500 0.4464 0.9996 0.1719
0.0615 44.1489 16600 0.4726 0.9982 0.1718
0.0615 44.4149 16700 0.4456 0.9980 0.1704
0.0615 44.6809 16800 0.4247 0.9980 0.1693
0.0615 44.9468 16900 0.4499 0.9986 0.1684
0.0524 45.2128 17000 0.4610 0.9988 0.1667
0.0524 45.4787 17100 0.4252 0.9982 0.1675
0.0524 45.7447 17200 0.4185 0.9980 0.1670
0.0524 46.0106 17300 0.4377 0.9980 0.1665
0.0524 46.2766 17400 0.4387 0.9992 0.1666
0.0466 46.5426 17500 0.4388 0.9986 0.1659
0.0466 46.8085 17600 0.4408 0.9986 0.1654
0.0466 47.0745 17700 0.4277 0.9984 0.1651
0.0466 47.3404 17800 0.4244 0.9986 0.1650
0.0466 47.6064 17900 0.4296 0.9978 0.1644
0.0393 47.8723 18000 0.4341 0.9984 0.1648
0.0393 48.1383 18100 0.4337 0.9982 0.1646
0.0393 48.4043 18200 0.4331 0.9978 0.1642
0.0393 48.6702 18300 0.4281 0.9980 0.1641
0.0393 48.9362 18400 0.4268 0.9982 0.1641
0.0373 49.2021 18500 0.4275 0.9982 0.1641
0.0373 49.4681 18600 0.4269 0.9982 0.1641
0.0373 49.7340 18700 0.4268 0.9978 0.1640
0.0373 50.0 18800 0.4271 0.9978 0.1642

Framework versions

  • Transformers 4.47.0.dev0
  • Pytorch 2.5.1+cu124
  • Datasets 3.1.0
  • Tokenizers 0.20.3