---
library_name: transformers
language:
- xh
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
- generated_from_trainer
datasets:
- NCHLT_speech_corpus
metrics:
- wer
model-index:
- name: facebook mms-1b-all xhosa - Beijuka Bruno
  results:
  - task:
      name: Automatic Speech Recognition
      type: automatic-speech-recognition
    dataset:
      name: NCHLT_speech_corpus/Xhosa
      type: NCHLT_speech_corpus
    metrics:
    - name: Wer
      type: wer
      value: 0.5971681747855033
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# facebook mms-1b-all xhosa - Beijuka Bruno

This model is a fine-tuned version of [facebook/mms-1b-all](https://huggingface.co/facebook/mms-1b-all) on the NCHLT_speech_corpus/Xhosa dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3580
- Model Preparation Time: 0.0192
- Wer: 0.5972
- Cer: 0.1361

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0003
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 100
- num_epochs: 100

### Training results

| Training Loss | Epoch   | Step  | Validation Loss | Model Preparation Time | Wer    | Cer    |
|:-------------:|:-------:|:-----:|:---------------:|:----------------------:|:------:|:------:|
| 14.2293       | 0.9976  | 316   | 0.2772          | 0.0192                 | 0.3588 | 0.0640 |
| 1.2546        | 1.9976  | 632   | 0.2685          | 0.0192                 | 0.3497 | 0.0616 |
| 1.1714        | 2.9976  | 948   | 0.2550          | 0.0192                 | 0.3326 | 0.0598 |
| 1.1021        | 3.9976  | 1264  | 0.2483          | 0.0192                 | 0.3212 | 0.0583 |
| 1.0443        | 4.9976  | 1580  | 0.2418          | 0.0192                 | 0.3073 | 0.0567 |
| 0.9824        | 5.9976  | 1896  | 0.2426          | 0.0192                 | 0.3003 | 0.0558 |
| 0.959         | 6.9976  | 2212  | 0.2345          | 0.0192                 | 0.2895 | 0.0547 |
| 0.9257        | 7.9976  | 2528  | 0.2365          | 0.0192                 | 0.2988 | 0.0551 |
| 0.8916        | 8.9976  | 2844  | 0.2294          | 0.0192                 | 0.2862 | 0.0536 |
| 0.8608        | 9.9976  | 3160  | 0.2263          | 0.0192                 | 0.2864 | 0.0537 |
| 0.8313        | 10.9976 | 3476  | 0.2297          | 0.0192                 | 0.2817 | 0.0532 |
| 0.8067        | 11.9976 | 3792  | 0.2269          | 0.0192                 | 0.2796 | 0.0524 |
| 0.7644        | 12.9976 | 4108  | 0.2264          | 0.0192                 | 0.2703 | 0.0513 |
| 0.7608        | 13.9976 | 4424  | 0.2264          | 0.0192                 | 0.2737 | 0.0515 |
| 0.7311        | 14.9976 | 4740  | 0.2293          | 0.0192                 | 0.2703 | 0.0520 |
| 0.7109        | 15.9976 | 5056  | 0.2236          | 0.0192                 | 0.2712 | 0.0505 |
| 0.7245        | 16.9976 | 5372  | 0.2333          | 0.0192                 | 0.2760 | 0.0516 |
| 0.6952        | 17.9976 | 5688  | 0.2256          | 0.0192                 | 0.2623 | 0.0504 |
| 0.6798        | 18.9976 | 6004  | 0.2245          | 0.0192                 | 0.2650 | 0.0497 |
| 0.6525        | 19.9976 | 6320  | 0.2254          | 0.0192                 | 0.2669 | 0.0499 |
| 0.6366        | 20.9976 | 6636  | 0.2208          | 0.0192                 | 0.2627 | 0.0495 |
| 0.6263        | 21.9976 | 6952  | 0.2197          | 0.0192                 | 0.2600 | 0.0496 |
| 0.618         | 22.9976 | 7268  | 0.2227          | 0.0192                 | 0.2562 | 0.0491 |
| 0.5989        | 23.9976 | 7584  | 0.2228          | 0.0192                 | 0.2566 | 0.0496 |
| 0.5869        | 24.9976 | 7900  | 0.2278          | 0.0192                 | 0.2621 | 0.0496 |
| 0.5667        | 25.9976 | 8216  | 0.2260          | 0.0192                 | 0.2591 | 0.0498 |
| 0.5607        | 26.9976 | 8532  | 0.2242          | 0.0192                 | 0.2642 | 0.0503 |
| 0.5563        | 27.9976 | 8848  | 0.2235          | 0.0192                 | 0.2528 | 0.0488 |
| 0.537         | 28.9976 | 9164  | 0.2354          | 0.0192                 | 0.2699 | 0.0513 |
| 0.53          | 29.9976 | 9480  | 0.2233          | 0.0192                 | 0.2541 | 0.0495 |
| 0.5143        | 30.9976 | 9796  | 0.2289          | 0.0192                 | 0.2524 | 0.0494 |
| 0.5118        | 31.9976 | 10112 | 0.2295          | 0.0192                 | 0.2596 | 0.0497 |
| 0.5041        | 32.9976 | 10428 | 0.2287          | 0.0192                 | 0.2547 | 0.0485 |
| 0.4918        | 33.9976 | 10744 | 0.2361          | 0.0192                 | 0.2536 | 0.0493 |
| 0.4825        | 34.9976 | 11060 | 0.2337          | 0.0192                 | 0.2562 | 0.0495 |
| 0.4758        | 35.9976 | 11376 | 0.2297          | 0.0192                 | 0.2496 | 0.0486 |
| 0.4525        | 36.9976 | 11692 | 0.2336          | 0.0192                 | 0.2501 | 0.0489 |
| 0.4564        | 37.9976 | 12008 | 0.2366          | 0.0192                 | 0.2602 | 0.0493 |
| 0.4555        | 38.9976 | 12324 | 0.2329          | 0.0192                 | 0.2515 | 0.0485 |
| 0.4471        | 39.9976 | 12640 | 0.2349          | 0.0192                 | 0.2505 | 0.0478 |
| 0.4324        | 40.9976 | 12956 | 0.2360          | 0.0192                 | 0.2526 | 0.0490 |
| 0.4219        | 41.9976 | 13272 | 0.2423          | 0.0192                 | 0.2541 | 0.0496 |
| 0.4172        | 42.9976 | 13588 | 0.2319          | 0.0192                 | 0.2448 | 0.0482 |
| 0.3976        | 43.9976 | 13904 | 0.2363          | 0.0192                 | 0.2460 | 0.0482 |
| 0.3991        | 44.9976 | 14220 | 0.2351          | 0.0192                 | 0.2663 | 0.0502 |
| 0.4055        | 45.9976 | 14536 | 0.2401          | 0.0192                 | 0.2507 | 0.0487 |
| 0.3889        | 46.9976 | 14852 | 0.2418          | 0.0192                 | 0.2496 | 0.0479 |
| 0.3931        | 47.9976 | 15168 | 0.2441          | 0.0192                 | 0.2475 | 0.0480 |
| 0.3651        | 48.9976 | 15484 | 0.2439          | 0.0192                 | 0.2549 | 0.0491 |
| 0.3707        | 49.9976 | 15800 | 0.2428          | 0.0192                 | 0.2511 | 0.0486 |
| 0.378         | 50.9976 | 16116 | 0.2371          | 0.0192                 | 0.2450 | 0.0479 |
| 0.3604        | 51.9976 | 16432 | 0.2462          | 0.0192                 | 0.2486 | 0.0483 |
| 0.3686        | 52.9976 | 16748 | 0.2457          | 0.0192                 | 0.2501 | 0.0484 |


### Framework versions

- Transformers 4.47.1
- Pytorch 2.1.0+cu118
- Datasets 3.2.0
- Tokenizers 0.21.0