metadata

license: apache-2.0
tags:
  - generated_from_trainer
  - summarization
datasets:
  - xsum
  - autoevaluate/xsum-sample
metrics:
  - rouge
duplicated_from: autoevaluate/summarization
model-index:
  - name: autoevaluate/summarization-not-evaluated
    results:
      - task:
          type: summarization
          name: Summarization
        dataset:
          name: autoevaluate/xsum-sample
          type: autoevaluate/xsum-sample
          config: autoevaluate--xsum-sample
          split: test
        metrics:
          - type: rouge
            value: 18.3975
            name: ROUGE-1
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWVjY2U1N2I3MmI0ZGVhMzQwN2E0NzY0OTk3NDBjNWZhODVkMDQ4ZDMwYzUwYmQ3MDYyZmFmMTM2NGUwNTMzOSIsInZlcnNpb24iOjF9.-1drSQmRK8GwHjrGWpmCzRLWT9NDfImYiLZEsO4doWMsyz7lik00-roc0wFDrp6JpFb5Uc3vvaJ4dg4SlU3pBw
          - type: rouge
            value: 3.122
            name: ROUGE-2
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNmMxMjlkNzE0NTdiMzg3MTViOGM5ZWRlM2E4ODA0NWM2Y2M5NTQwYTI1NTUxMWJmNmIwNjcwNzYyMjZhNDliMSIsInZlcnNpb24iOjF9._qnIJsSZgY-Cr3lfUGOjsRakjg0L8HZmpI_bZnf2FJxJCOIDd4RjPAsibTdZLL5u3nJK72CstIvzLyAzPEdKCA
          - type: rouge
            value: 14.8693
            name: ROUGE-L
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWZlNDkxODUwYzgxYzMzYzNlYWJlNDQyYTkxM2FjZmYyYjk1MDBlYmVkNGY3OWQxOTBkOGI2NzYwZTlhMGUyZSIsInZlcnNpb24iOjF9.eV8j4NkoIp2vhbXZGE6Np_DMc7zBB5gycg_wCcejCJ72vN_Pog8K_2Ltu2N-vtStJrM_3QmbZMnWMS0OgCDlCA
          - type: rouge
            value: 14.8537
            name: ROUGE-LSUM
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMjBjOWVhOTNmN2NiYjQyZmYwMzc1NTRjODIwYWY5MDIzYWU2MmQ0N2MxZjUyNDU4MDEzZWM0N2FjNGQxZmQ3NiIsInZlcnNpb24iOjF9.tUF5mFbt4iw3NXX8zmxAWLGKxiOJ0N6pMuot2EiPnGsCKv_s6zKf-Akc1NA89TjLGc8lMvUkKJyFbqIU1n9eBg
          - type: loss
            value: 3.0104541778564453
            name: loss
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTM5NDI0ZjMxOWIwZGFhZTkxMzNkOGRiNjM4YTAxODkyMmZhOGYzNjQ0MTVmZWEyZWU5MTRlODM5YWYzYmIxYiIsInZlcnNpb24iOjF9.uCcjMNr5q_pL6HbVF5cwkh9yO2ni5GieWg5oWjhFWv5ZslnB7OqskPdLt_P3YMYQRUkYRamzdYQYh5qJB7oYCA
          - type: gen_len
            value: 18.05
            name: gen_len
            verified: true
            verifyToken: >-
              eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYjZhNmYxNjYxNTQ4MzA0YTNhYTIwMzVhZTNlMGE5MmI5M2JkZDY4MzA3YTY1Zjk4NjRhNjY5NjcxYjcwYmU3NSIsInZlcnNpb24iOjF9.JKyvAJB5uFL6srWzoj9P_OIK9tnT7GW8mGPFVNrY6SxjbupdIry2iPrM5YpBdt6sLeS5_DgdP80OGGjwfwBiBQ

summarization

This model is a fine-tuned version of t5-small on the xsum dataset. It achieves the following results on the evaluation set:

Loss: 2.6690
Rouge1: 23.9405
Rouge2: 5.0879
Rougel: 18.4981
Rougelsum: 18.5032
Gen Len: 18.7376

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
training_steps: 1000
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.9249	0.08	1000	2.6690	23.9405	5.0879	18.4981	18.5032	18.7376

Framework versions

Transformers 4.19.2
Pytorch 1.11.0+cu113
Datasets 2.2.2
Tokenizers 0.12.1