hubert-base-ls960-finetuned-ic-slurp-wt_init

This model is a fine-tuned version of facebook/hubert-base-ls960 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1377
• Accuracy: 0.4604

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 96
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
3.9613	1.0	527	3.8944	0.0803
3.7817	2.0	1055	3.7275	0.0910
3.6357	3.0	1582	3.5410	0.1308
3.4527	4.0	2110	3.3426	0.1676
3.0715	5.0	2637	3.0751	0.2331
2.9153	6.0	3165	2.8168	0.2969
2.5333	7.0	3692	2.6229	0.3375
2.3807	8.0	4220	2.5673	0.3620
2.181	9.0	4747	2.4933	0.3835
1.9118	10.0	5275	2.4411	0.4046
1.9015	11.0	5802	2.4254	0.4126
1.5811	12.0	6330	2.4216	0.4275
1.491	13.0	6857	2.4833	0.4284
1.3697	14.0	7385	2.5243	0.4368
1.1232	15.0	7912	2.5944	0.4309
1.1071	16.0	8440	2.6475	0.4317
0.9439	17.0	8967	2.6379	0.4449
0.917	18.0	9495	2.7438	0.4468
0.7628	19.0	10022	2.7671	0.4513
0.7642	20.0	10550	2.8993	0.4418
0.6716	21.0	11077	2.9354	0.4472
0.6166	22.0	11605	2.9961	0.4510
0.4819	23.0	12132	3.0959	0.4451
0.5903	24.0	12660	3.0542	0.4557
0.515	25.0	13187	3.0723	0.4589
0.518	26.0	13715	3.1377	0.4604
0.3902	27.0	14242	3.2230	0.4524
0.4825	28.0	14770	3.2925	0.4583
0.29	29.0	15297	3.4027	0.4498
0.2789	30.0	15825	3.3573	0.4598
0.3202	31.0	16352	3.4381	0.4542

Framework versions

• Transformers 4.37.2
• Pytorch 2.2.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2