metadata

license: apache-2.0
tags:
  - generated_from_trainer
datasets:
  - govreport-summarization
metrics:
  - rouge
model-index:
  - name: led-large-16384-govreport
    results:
      - task:
          name: Sequence-to-sequence Language Modeling
          type: text2text-generation
        dataset:
          name: govreport-summarization
          type: govreport-summarization
          config: document
          split: validation
          args: document
        metrics:
          - name: Rouge1
            type: rouge
            value: 0.5194151586540673

led-large-16384-govreport

This model is a fine-tuned version of allenai/led-base-16384 on the govreport-summarization dataset. It achieves the following results on the evaluation set:

Loss: 1.7624
Rouge1: 0.5194
Rouge2: 0.2107
Rougel: 0.2437
Rougelsum: 0.2437

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 1
eval_batch_size: 1
seed: 42
gradient_accumulation_steps: 64
total_train_batch_size: 64
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
1.8152	3.65	500	1.7956	0.5095	0.2040	0.2382	0.2381
1.6981	3.66	1000	1.7624	0.5194	0.2107	0.2437	0.2437

Framework versions

Transformers 4.30.2
Pytorch 1.10.0+cu102
Datasets 2.13.1
Tokenizers 0.13.3