metadata
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: t5-small-github-repo-tag-generation
results: []
t5-small-github-repo-tag-generation
This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.5366
- Rouge1: 20.3802
- Rouge2: 4.7145
- Rougel: 18.411
- Rougelsum: 18.3921
- Gen Len: 19.0
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
|---|---|---|---|---|---|---|---|---|
| 1.4594 | 1.0 | 66 | 0.9943 | 3.9283 | 0.0 | 3.7797 | 3.7332 | 19.0 |
| 0.9732 | 2.0 | 132 | 0.8077 | 1.3301 | 0.0 | 1.2269 | 1.2141 | 19.0 |
| 0.8414 | 3.0 | 198 | 0.7334 | 4.3519 | 0.3755 | 4.2376 | 4.2974 | 19.0 |
| 0.7843 | 4.0 | 264 | 0.6971 | 17.7535 | 1.5237 | 16.2304 | 16.1943 | 19.0 |
| 0.7463 | 5.0 | 330 | 0.6655 | 19.3128 | 2.2678 | 16.5655 | 16.5492 | 19.0 |
| 0.7182 | 6.0 | 396 | 0.6412 | 19.8633 | 3.3647 | 17.5046 | 17.4973 | 19.0 |
| 0.6959 | 7.0 | 462 | 0.6286 | 20.274 | 3.1415 | 17.8527 | 17.9039 | 19.0 |
| 0.6793 | 8.0 | 528 | 0.6121 | 19.9915 | 3.5137 | 18.1718 | 18.2166 | 19.0 |
| 0.6666 | 9.0 | 594 | 0.6021 | 20.073 | 3.6482 | 18.3466 | 18.3829 | 19.0 |
| 0.6561 | 10.0 | 660 | 0.5923 | 20.6227 | 3.8946 | 18.6466 | 18.694 | 19.0 |
| 0.6471 | 11.0 | 726 | 0.5867 | 20.3914 | 4.1584 | 18.5822 | 18.5744 | 19.0 |
| 0.638 | 12.0 | 792 | 0.5776 | 20.9846 | 4.6568 | 18.8597 | 18.8979 | 19.0 |
| 0.6343 | 13.0 | 858 | 0.5715 | 19.7497 | 4.4363 | 18.262 | 18.2526 | 19.0 |
| 0.6267 | 14.0 | 924 | 0.5670 | 19.5839 | 4.2251 | 17.9116 | 17.9491 | 19.0 |
| 0.6211 | 15.0 | 990 | 0.5602 | 19.4954 | 4.4755 | 17.9726 | 17.9203 | 19.0 |
| 0.6169 | 16.0 | 1056 | 0.5557 | 19.6247 | 4.2106 | 18.0385 | 18.0922 | 19.0 |
| 0.6103 | 17.0 | 1122 | 0.5532 | 19.8744 | 4.4672 | 18.1794 | 18.1689 | 19.0 |
| 0.6062 | 18.0 | 1188 | 0.5505 | 19.8744 | 4.3368 | 18.1794 | 18.1689 | 19.0 |
| 0.6016 | 19.0 | 1254 | 0.5461 | 20.083 | 4.3446 | 18.4196 | 18.383 | 19.0 |
| 0.5987 | 20.0 | 1320 | 0.5426 | 20.029 | 4.5884 | 18.189 | 18.1371 | 19.0 |
| 0.5968 | 21.0 | 1386 | 0.5407 | 19.5394 | 4.0665 | 17.8456 | 17.7709 | 19.0 |
| 0.5957 | 22.0 | 1452 | 0.5387 | 19.6368 | 4.5837 | 18.0482 | 18.0676 | 19.0 |
| 0.5923 | 23.0 | 1518 | 0.5366 | 20.0744 | 4.8413 | 18.3513 | 18.2666 | 19.0 |
| 0.588 | 24.0 | 1584 | 0.5366 | 20.3802 | 4.7145 | 18.411 | 18.3921 | 19.0 |
Framework versions
- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.0
- Tokenizers 0.13.2