nhi_heldout-speaker-exp_DHM506_mms-1b-nhi-adapterft

This model is a fine-tuned version of facebook/mms-1b-all on the audiofolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3737
• Wer: 0.3136
• Cer: 0.0801

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.1381	1.3605	200	0.4997	0.4920	0.1236
0.9069	2.7211	400	0.4093	0.4282	0.1088
0.8151	4.0816	600	0.3993	0.4285	0.1068
0.7516	5.4422	800	0.3700	0.4002	0.1002
0.7257	6.8027	1000	0.3502	0.3875	0.0953
0.7108	8.1633	1200	0.3546	0.3896	0.0986
0.6728	9.5238	1400	0.3421	0.3872	0.0965
0.6294	10.8844	1600	0.3301	0.3576	0.0891
0.6265	12.2449	1800	0.3371	0.3733	0.0897
0.6094	13.6054	2000	0.3329	0.3690	0.0909
0.5879	14.9660	2200	0.3274	0.3606	0.0907
0.5685	16.3265	2400	0.3415	0.3875	0.0928
0.5527	17.6871	2600	0.3231	0.3418	0.0859
0.5468	19.0476	2800	0.3317	0.3777	0.0923
0.5211	20.4082	3000	0.3327	0.3828	0.0913
0.5177	21.7687	3200	0.3194	0.3513	0.0869
0.535	23.1293	3400	0.3200	0.3538	0.0875
0.4884	24.4898	3600	0.3097	0.3446	0.0868
0.5036	25.8503	3800	0.3184	0.3456	0.0864
0.4815	27.2109	4000	0.3259	0.3443	0.0852
0.4663	28.5714	4200	0.3354	0.3541	0.0871
0.4629	29.9320	4400	0.3136	0.3478	0.0864
0.4594	31.2925	4600	0.3211	0.3356	0.0839
0.4369	32.6531	4800	0.3181	0.3372	0.0845
0.4373	34.0136	5000	0.3189	0.3353	0.0835
0.4147	35.3741	5200	0.3284	0.3288	0.0842
0.429	36.7347	5400	0.3331	0.3332	0.0861
0.4124	38.0952	5600	0.3321	0.3231	0.0826
0.3899	39.4558	5800	0.3312	0.3351	0.0829
0.3956	40.8163	6000	0.3505	0.3253	0.0835
0.3897	42.1769	6200	0.3458	0.3318	0.0851
0.3865	43.5374	6400	0.3334	0.3361	0.0844
0.3847	44.8980	6600	0.3327	0.3402	0.0836
0.3656	46.2585	6800	0.3351	0.3250	0.0809
0.3766	47.6190	7000	0.3339	0.3264	0.0825
0.3672	48.9796	7200	0.3536	0.3337	0.0836
0.3583	50.3401	7400	0.3410	0.3272	0.0823
0.3418	51.7007	7600	0.3283	0.3226	0.0805
0.3351	53.0612	7800	0.3414	0.3247	0.0810
0.3479	54.4218	8000	0.3354	0.3264	0.0820
0.334	55.7823	8200	0.3438	0.3196	0.0814
0.3269	57.1429	8400	0.3499	0.3285	0.0816
0.3206	58.5034	8600	0.3435	0.3315	0.0820
0.3131	59.8639	8800	0.3508	0.3255	0.0802
0.3124	61.2245	9000	0.3482	0.3253	0.0811
0.3143	62.5850	9200	0.3362	0.3302	0.0807
0.3161	63.9456	9400	0.3408	0.3177	0.0790
0.2925	65.3061	9600	0.3479	0.3250	0.0815
0.2853	66.6667	9800	0.3583	0.3247	0.0812
0.2917	68.0272	10000	0.3569	0.3307	0.0825
0.3025	69.3878	10200	0.3545	0.3296	0.0821
0.2873	70.7483	10400	0.3547	0.3217	0.0799
0.2835	72.1088	10600	0.3570	0.3182	0.0806
0.2867	73.4694	10800	0.3598	0.3144	0.0808
0.2859	74.8299	11000	0.3587	0.3188	0.0813
0.276	76.1905	11200	0.3578	0.3139	0.0800
0.2795	77.5510	11400	0.3727	0.3220	0.0818
0.2699	78.9116	11600	0.3741	0.3155	0.0808
0.2556	80.2721	11800	0.3747	0.3174	0.0806
0.2593	81.6327	12000	0.3719	0.3275	0.0812
0.2554	82.9932	12200	0.3664	0.3209	0.0804
0.2535	84.3537	12400	0.3820	0.3264	0.0817
0.2504	85.7143	12600	0.3780	0.3231	0.0815
0.2459	87.0748	12800	0.3730	0.3236	0.0821
0.2482	88.4354	13000	0.3715	0.3209	0.0797
0.2443	89.7959	13200	0.3783	0.3190	0.0809
0.244	91.1565	13400	0.3641	0.3234	0.0813
0.2433	92.5170	13600	0.3763	0.3207	0.0817
0.2435	93.8776	13800	0.3764	0.3150	0.0799
0.2285	95.2381	14000	0.3762	0.3193	0.0807
0.2329	96.5986	14200	0.3751	0.3179	0.0803
0.2368	97.9592	14400	0.3750	0.3152	0.0800
0.2462	99.3197	14600	0.3737	0.3136	0.0801

Framework versions

• Transformers 4.41.2
• Pytorch 2.4.0
• Datasets 3.2.0
• Tokenizers 0.19.1