t5_small_autotagging

This model is a fine-tuned version of RevoltronTechno/t5_small_context_tagging on the autotagging dataset. It achieves the following results on the evaluation set:

• Loss: 0.5257

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
0.6798	1.0	1250	0.6040
0.6228	2.0	2500	0.5913
0.6253	3.0	3750	0.5829
0.6079	4.0	5000	0.5746
0.5902	5.0	6250	0.5688
0.5815	6.0	7500	0.5616
0.5669	7.0	8750	0.5576
0.5629	8.0	10000	0.5522
0.5798	9.0	11250	0.5500
0.5639	10.0	12500	0.5462
0.5458	11.0	13750	0.5428
0.5427	12.0	15000	0.5421
0.5198	13.0	16250	0.5399
0.5077	14.0	17500	0.5381
0.5267	15.0	18750	0.5366
0.4932	16.0	20000	0.5354
0.5046	17.0	21250	0.5342
0.5011	18.0	22500	0.5340
0.5029	19.0	23750	0.5325
0.4876	20.0	25000	0.5305
0.4926	21.0	26250	0.5289
0.4817	22.0	27500	0.5289
0.488	23.0	28750	0.5280
0.5012	24.0	30000	0.5283
0.4789	25.0	31250	0.5269
0.4913	26.0	32500	0.5275
0.4587	27.0	33750	0.5272
0.4718	28.0	35000	0.5266
0.4698	29.0	36250	0.5260
0.4873	30.0	37500	0.5256
0.4439	31.0	38750	0.5269
0.4794	32.0	40000	0.5263
0.4538	33.0	41250	0.5270
0.4613	34.0	42500	0.5250
0.4622	35.0	43750	0.5257
0.4567	36.0	45000	0.5257
0.4589	37.0	46250	0.5250
0.4508	38.0	47500	0.5252
0.4562	39.0	48750	0.5251
0.4405	40.0	50000	0.5257
0.4367	41.0	51250	0.5261
0.4407	42.0	52500	0.5256
0.4647	43.0	53750	0.5254
0.444	44.0	55000	0.5249
0.4582	45.0	56250	0.5254
0.4523	46.0	57500	0.5258
0.4435	47.0	58750	0.5255
0.4547	48.0	60000	0.5259
0.447	49.0	61250	0.5257
0.4272	50.0	62500	0.5257

Framework versions

• Transformers 4.46.2
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3

Evaluation Metrics

The model was evaluated using the following metrics:

Metric	Value	Percentage
ROUGE-1	0.513954	51.40%
ROUGE-2	0.251370	25.14%
ROUGE-L	0.465847	46.58%
BLEU Score	0.180733	18.07%

Metric Descriptions:

• ROUGE-1: Measures the overlap of unigrams between the generated text and reference text.
• ROUGE-2: Measures the overlap of bigrams between the generated text and reference text.
• ROUGE-L: Measures the longest common subsequence between the generated text and reference text.
• BLEU Score: Measures the precision of n-grams in the generated text compared to the reference text.