README.md

metadata

library_name: transformers
language:
  - xh
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
  - generated_from_trainer
datasets:
  - NCHLT_speech_corpus
metrics:
  - wer
model-index:
  - name: facebook mms-1b-all xhosa - Beijuka Bruno
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: NCHLT_speech_corpus/Xhosa
          type: NCHLT_speech_corpus
        metrics:
          - name: Wer
            type: wer
            value: 0.4469970462750246

facebook mms-1b-all xhosa - Beijuka Bruno

This model is a fine-tuned version of facebook/mms-1b-all on the NCHLT_speech_corpus/Xhosa dataset. It achieves the following results on the evaluation set:

• Loss: 0.2822
• Model Preparation Time: 0.0163
• Wer: 0.4470
• Cer: 0.0841

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Wer	Cer
53.6849	0.9955	167	0.2975	0.0163	0.3955	0.0660
2.6944	1.9955	334	0.2671	0.0163	0.3716	0.0612
2.4353	2.9955	501	0.2543	0.0163	0.3468	0.0580
2.2597	3.9955	668	0.2459	0.0163	0.3445	0.0565
2.1704	4.9955	835	0.2392	0.0163	0.3251	0.0547
2.0144	5.9955	1002	0.2353	0.0163	0.3160	0.0539
1.9278	6.9955	1169	0.2350	0.0163	0.3143	0.0539
1.849	7.9955	1336	0.2312	0.0163	0.3039	0.0520
1.811	8.9955	1503	0.2289	0.0163	0.3065	0.0530
1.7173	9.9955	1670	0.2311	0.0163	0.2967	0.0522
1.6504	10.9955	1837	0.2205	0.0163	0.2960	0.0511
1.5737	11.9955	2004	0.2227	0.0163	0.2904	0.0506
1.5547	12.9955	2171	0.2218	0.0163	0.2836	0.0502
1.5517	13.9955	2338	0.2236	0.0163	0.2917	0.0507
1.5031	14.9955	2505	0.2162	0.0163	0.2826	0.0493
1.4097	15.9955	2672	0.2187	0.0163	0.2773	0.0492
1.4143	16.9955	2839	0.2176	0.0163	0.2809	0.0485
1.36	17.9955	3006	0.2200	0.0163	0.2675	0.0479
1.3312	18.9955	3173	0.2163	0.0163	0.2672	0.0482
1.3369	19.9955	3340	0.2199	0.0163	0.2678	0.0481
1.3036	20.9955	3507	0.2224	0.0163	0.2714	0.0482
1.2558	21.9955	3674	0.2244	0.0163	0.2656	0.0478
1.2058	22.9955	3841	0.2192	0.0163	0.2642	0.0481
1.167	23.9955	4008	0.2225	0.0163	0.2561	0.0471
1.1849	24.9955	4175	0.2275	0.0163	0.2610	0.0473
1.1378	25.9955	4342	0.2246	0.0163	0.2610	0.0474
1.1095	26.9955	4509	0.2295	0.0163	0.2538	0.0464
1.1042	27.9955	4676	0.2243	0.0163	0.2518	0.0462
1.0537	28.9955	4843	0.2293	0.0163	0.2531	0.0467
1.0335	29.9955	5010	0.2264	0.0163	0.2531	0.0456
1.0453	30.9955	5177	0.2232	0.0163	0.2525	0.0452
1.0099	31.9955	5344	0.2285	0.0163	0.2551	0.0467
0.9826	32.9955	5511	0.2345	0.0163	0.2570	0.0470
0.9615	33.9955	5678	0.2361	0.0163	0.2587	0.0471
0.9583	34.9955	5845	0.2340	0.0163	0.2528	0.0452
0.9421	35.9955	6012	0.2339	0.0163	0.2443	0.0452
0.9268	36.9955	6179	0.2350	0.0163	0.2518	0.0455
0.9003	37.9955	6346	0.2380	0.0163	0.2472	0.0451
0.9068	38.9955	6513	0.2424	0.0163	0.2515	0.0459
0.8845	39.9955	6680	0.2466	0.0163	0.2525	0.0462
0.88	40.9955	6847	0.2423	0.0163	0.2387	0.0441
0.8683	41.9955	7014	0.2448	0.0163	0.2528	0.0463
0.8535	42.9955	7181	0.2498	0.0163	0.2492	0.0455
0.8413	43.9955	7348	0.2431	0.0163	0.2511	0.0463
0.8147	44.9955	7515	0.2416	0.0163	0.2426	0.0449
0.8062	45.9955	7682	0.2483	0.0163	0.2479	0.0455
0.7876	46.9955	7849	0.2477	0.0163	0.2531	0.0463
0.8108	47.9955	8016	0.2469	0.0163	0.2462	0.0454
0.7689	48.9955	8183	0.2539	0.0163	0.2489	0.0460
0.7609	49.9955	8350	0.2535	0.0163	0.2453	0.0446
0.7442	50.9955	8517	0.2603	0.0163	0.2485	0.0459

Framework versions

• Transformers 4.47.1
• Pytorch 2.1.0+cu118
• Datasets 3.2.0
• Tokenizers 0.21.0