metadata

library_name: transformers
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
  - automatic-speech-recognition
  - bigcgen
  - mms
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: mms-1b-bigcgen-female-30hrs-model
    results: []

mms-1b-bigcgen-female-30hrs-model

This model is a fine-tuned version of facebook/mms-1b-all on the BIGCGEN - BEM dataset. It achieves the following results on the evaluation set:

Loss: inf
Wer: 0.5369

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0003
train_batch_size: 4
eval_batch_size: 4
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 8
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
training_steps: 2500
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
14.2597	0.0501	100	inf	1.0025
6.2197	0.1002	200	inf	0.9946
5.2752	0.1503	300	inf	1.0316
2.3626	0.2004	400	inf	0.5998
1.7448	0.2504	500	inf	0.5814
1.6955	0.3005	600	inf	0.5709
1.6841	0.3506	700	inf	0.5596
1.693	0.4007	800	inf	0.5639
1.688	0.4508	900	inf	0.5555
1.5718	0.5009	1000	inf	0.5476
1.5855	0.5510	1100	inf	0.5482
1.4783	0.6011	1200	inf	0.5471
1.5198	0.6511	1300	inf	0.5476
1.4941	0.7012	1400	inf	0.5469
1.5916	0.7513	1500	inf	0.5426
1.4683	0.8014	1600	inf	0.5460
1.486	0.8515	1700	inf	0.5528
1.4353	0.9016	1800	inf	0.5435
1.6166	0.9517	1900	inf	0.5541
1.531	1.0015	2000	inf	0.5535
1.5441	1.0516	2100	inf	0.5478
1.3459	1.1017	2200	inf	0.5274
1.357	1.1518	2300	inf	0.5269
1.4464	1.2019	2400	inf	0.5226
1.4326	1.2519	2500	inf	0.5369

Framework versions

Transformers 4.47.1
Pytorch 2.5.1+cu124
Datasets 3.2.0
Tokenizers 0.21.0