---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- rouge
model-index:
- name: t5-small-github-repo-tag-generation
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# t5-small-github-repo-tag-generation

This model is a fine-tuned version of [t5-small](https://huggingface.co/t5-small) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5366
- Rouge1: 20.3802
- Rouge2: 4.7145
- Rougel: 18.411
- Rougelsum: 18.3921
- Gen Len: 19.0

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 30
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Rouge1  | Rouge2 | Rougel  | Rougelsum | Gen Len |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:------:|:-------:|:---------:|:-------:|
| 1.4594        | 1.0   | 66   | 0.9943          | 3.9283  | 0.0    | 3.7797  | 3.7332    | 19.0    |
| 0.9732        | 2.0   | 132  | 0.8077          | 1.3301  | 0.0    | 1.2269  | 1.2141    | 19.0    |
| 0.8414        | 3.0   | 198  | 0.7334          | 4.3519  | 0.3755 | 4.2376  | 4.2974    | 19.0    |
| 0.7843        | 4.0   | 264  | 0.6971          | 17.7535 | 1.5237 | 16.2304 | 16.1943   | 19.0    |
| 0.7463        | 5.0   | 330  | 0.6655          | 19.3128 | 2.2678 | 16.5655 | 16.5492   | 19.0    |
| 0.7182        | 6.0   | 396  | 0.6412          | 19.8633 | 3.3647 | 17.5046 | 17.4973   | 19.0    |
| 0.6959        | 7.0   | 462  | 0.6286          | 20.274  | 3.1415 | 17.8527 | 17.9039   | 19.0    |
| 0.6793        | 8.0   | 528  | 0.6121          | 19.9915 | 3.5137 | 18.1718 | 18.2166   | 19.0    |
| 0.6666        | 9.0   | 594  | 0.6021          | 20.073  | 3.6482 | 18.3466 | 18.3829   | 19.0    |
| 0.6561        | 10.0  | 660  | 0.5923          | 20.6227 | 3.8946 | 18.6466 | 18.694    | 19.0    |
| 0.6471        | 11.0  | 726  | 0.5867          | 20.3914 | 4.1584 | 18.5822 | 18.5744   | 19.0    |
| 0.638         | 12.0  | 792  | 0.5776          | 20.9846 | 4.6568 | 18.8597 | 18.8979   | 19.0    |
| 0.6343        | 13.0  | 858  | 0.5715          | 19.7497 | 4.4363 | 18.262  | 18.2526   | 19.0    |
| 0.6267        | 14.0  | 924  | 0.5670          | 19.5839 | 4.2251 | 17.9116 | 17.9491   | 19.0    |
| 0.6211        | 15.0  | 990  | 0.5602          | 19.4954 | 4.4755 | 17.9726 | 17.9203   | 19.0    |
| 0.6169        | 16.0  | 1056 | 0.5557          | 19.6247 | 4.2106 | 18.0385 | 18.0922   | 19.0    |
| 0.6103        | 17.0  | 1122 | 0.5532          | 19.8744 | 4.4672 | 18.1794 | 18.1689   | 19.0    |
| 0.6062        | 18.0  | 1188 | 0.5505          | 19.8744 | 4.3368 | 18.1794 | 18.1689   | 19.0    |
| 0.6016        | 19.0  | 1254 | 0.5461          | 20.083  | 4.3446 | 18.4196 | 18.383    | 19.0    |
| 0.5987        | 20.0  | 1320 | 0.5426          | 20.029  | 4.5884 | 18.189  | 18.1371   | 19.0    |
| 0.5968        | 21.0  | 1386 | 0.5407          | 19.5394 | 4.0665 | 17.8456 | 17.7709   | 19.0    |
| 0.5957        | 22.0  | 1452 | 0.5387          | 19.6368 | 4.5837 | 18.0482 | 18.0676   | 19.0    |
| 0.5923        | 23.0  | 1518 | 0.5366          | 20.0744 | 4.8413 | 18.3513 | 18.2666   | 19.0    |
| 0.588         | 24.0  | 1584 | 0.5366          | 20.3802 | 4.7145 | 18.411  | 18.3921   | 19.0    |


### Framework versions

- Transformers 4.26.1
- Pytorch 1.13.1+cu116
- Datasets 2.10.0
- Tokenizers 0.13.2