mt5-base-finetuned-sumeczech

This model is a fine-tuned version of google/mt5-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.9291
  • Rouge1: 15.9842
  • Rouge2: 5.0275
  • Rougel: 12.6308
  • Rougelsum: 14.0073

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 8

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
2.6446 1.0 108450 2.4043 13.4797 3.1596 10.6012 11.798
2.3864 2.0 216900 2.3327 13.955 3.387 10.9208 12.165
2.3381 3.0 325350 2.2699 14.2671 3.5872 11.1539 12.4443
2.2583 4.0 433800 2.2085 14.5162 3.9249 11.4167 12.697
2.178 5.0 542250 2.1429 14.8376 4.1524 11.6426 12.9856
2.0847 6.0 650700 2.0678 15.0717 4.3497 11.8584 13.1779
1.9676 7.0 759150 1.9866 15.7074 4.7106 12.3935 13.7652
1.8196 8.0 867600 1.9291 15.9842 5.0275 12.6308 14.0073

Framework versions

  • Transformers 4.34.1
  • Pytorch 2.1.0+cu118
  • Datasets 2.14.6
  • Tokenizers 0.14.1
Downloads last month
728
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for tranv/mt5-base-finetuned-sumeczech

Base model

google/mt5-base
Finetuned
(168)
this model