metadata

license: apache-2.0
tags:
  - generated_from_trainer
datasets:
  - winograd_wsc
metrics:
  - rouge
widget:
  - text: Sam has a Parker pen. He loves writing with it.
    example_title: Example 1
  - text: >-
      Coronavirus quickly spread worldwide in 2020. The virus mostly affects
      elderly people. They can easily catch it.
    example_title: Example 2
  - text: >-
      First, the manager evaluates the candidates. Afterwards, he notifies the
      candidates regarding the evaluation.
    example_title: Example 3
base_model: google/flan-t5-small
model-index:
  - name: flan-t5-small-coref
    results:
      - task:
          type: text2text-generation
          name: Sequence-to-sequence Language Modeling
        dataset:
          name: winograd_wsc
          type: winograd_wsc
          config: wsc285
          split: test
          args: wsc285
        metrics:
          - type: rouge
            value: 0.906
            name: Rouge1

flan-t5-small-coref

This model is a fine-tuned version of google/flan-t5-small on the winograd_wsc dataset.

The model was trained on the task of coreference resolution.

It achieves the following results on the evaluation set:

Loss: 0.5656
Rouge1: 0.906
Rouge2: 0.8192
Rougel: 0.9016
Rougelsum: 0.9026
Gen Len: 23.1724

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	16	1.0901	0.6849	0.561	0.6734	0.6746	18.4483
No log	2.0	32	0.9083	0.8512	0.7509	0.8438	0.8437	21.1379
No log	3.0	48	0.8132	0.8638	0.7728	0.8588	0.8595	21.8276
No log	4.0	64	0.7590	0.8786	0.7842	0.8744	0.876	22.2069
No log	5.0	80	0.7225	0.8846	0.7928	0.8805	0.8817	22.3793
No log	6.0	96	0.6920	0.886	0.7942	0.8821	0.8827	22.4483
No log	7.0	112	0.6660	0.8861	0.7922	0.8816	0.8827	22.5172
No log	8.0	128	0.6470	0.8879	0.7953	0.8836	0.8849	22.6897
No log	9.0	144	0.6318	0.8968	0.806	0.8923	0.8933	23.069
No log	10.0	160	0.6160	0.8968	0.806	0.8923	0.8933	23.069
No log	11.0	176	0.6055	0.9056	0.822	0.9014	0.9021	23.1724
No log	12.0	192	0.5962	0.9056	0.822	0.9014	0.9021	23.1724
No log	13.0	208	0.5884	0.9074	0.8246	0.9033	0.9042	23.2069
No log	14.0	224	0.5825	0.9049	0.8182	0.9005	0.9016	23.2414
No log	15.0	240	0.5769	0.9049	0.8182	0.9005	0.9016	23.2414
No log	16.0	256	0.5727	0.903	0.8132	0.8991	0.8997	23.1724
No log	17.0	272	0.5698	0.906	0.8192	0.9016	0.9026	23.1724
No log	18.0	288	0.5673	0.906	0.8192	0.9016	0.9026	23.1724
No log	19.0	304	0.5661	0.906	0.8192	0.9016	0.9026	23.1724
No log	20.0	320	0.5656	0.906	0.8192	0.9016	0.9026	23.1724

Framework versions

Transformers 4.25.1
Pytorch 1.13.0+cu116
Datasets 2.7.1
Tokenizers 0.13.2