--- library_name: transformers license: apache-2.0 base_model: google/long-t5-tglobal-base tags: - generated_from_trainer metrics: - rouge model-index: - name: long_t5_3 results: [] --- # long_t5_3 This model is a fine-tuned version of [google/long-t5-tglobal-base](https://huggingface.co/google/long-t5-tglobal-base) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 2.1612 - Rouge1: 0.5309 - Rouge2: 0.3406 - Rougel: 0.4779 - Rougelsum: 0.4778 - Gen Len: 30.6175 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 0.0001 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 30 ### Training results | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len | |:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:| | 2.0161 | 1.0 | 1000 | 1.5665 | 0.4911 | 0.3059 | 0.4451 | 0.4451 | 25.5255 | | 1.7658 | 2.0 | 2000 | 1.5150 | 0.5026 | 0.3142 | 0.4559 | 0.4557 | 26.8015 | | 1.5969 | 3.0 | 3000 | 1.5031 | 0.51 | 0.3238 | 0.4628 | 0.4626 | 26.0075 | | 1.4638 | 4.0 | 4000 | 1.5048 | 0.5189 | 0.3348 | 0.4724 | 0.4724 | 26.878 | | 1.3675 | 5.0 | 5000 | 1.5363 | 0.5233 | 0.3369 | 0.4769 | 0.477 | 27.204 | | 1.249 | 6.0 | 6000 | 1.5550 | 0.5206 | 0.3376 | 0.4762 | 0.4759 | 25.569 | | 1.1861 | 7.0 | 7000 | 1.5511 | 0.5283 | 0.3444 | 0.4825 | 0.4824 | 26.8355 | | 1.0985 | 8.0 | 8000 | 1.5838 | 0.5284 | 0.342 | 0.4792 | 0.4792 | 28.631 | | 1.0178 | 9.0 | 9000 | 1.6231 | 0.5331 | 0.3451 | 0.4827 | 0.4828 | 28.7125 | | 0.9649 | 10.0 | 10000 | 1.6392 | 0.5262 | 0.3384 | 0.4762 | 0.4762 | 29.0855 | | 0.9069 | 11.0 | 11000 | 1.6758 | 0.5307 | 0.3421 | 0.4808 | 0.4804 | 28.9355 | | 0.8472 | 12.0 | 12000 | 1.7137 | 0.5304 | 0.3458 | 0.481 | 0.4809 | 29.29 | | 0.8087 | 13.0 | 13000 | 1.7478 | 0.5287 | 0.342 | 0.4789 | 0.4786 | 29.5185 | | 0.773 | 14.0 | 14000 | 1.7628 | 0.5302 | 0.3436 | 0.4801 | 0.4801 | 29.725 | | 0.7271 | 15.0 | 15000 | 1.8112 | 0.5293 | 0.3418 | 0.4789 | 0.4786 | 30.188 | | 0.6919 | 16.0 | 16000 | 1.8520 | 0.5293 | 0.342 | 0.4778 | 0.4778 | 30.4125 | | 0.665 | 17.0 | 17000 | 1.8738 | 0.5341 | 0.3432 | 0.4821 | 0.482 | 29.534 | | 0.6242 | 18.0 | 18000 | 1.9228 | 0.5314 | 0.3439 | 0.4793 | 0.4792 | 29.2675 | | 0.6024 | 19.0 | 19000 | 1.9288 | 0.535 | 0.347 | 0.4824 | 0.4823 | 29.852 | | 0.5791 | 20.0 | 20000 | 1.9614 | 0.531 | 0.3417 | 0.4793 | 0.4791 | 29.754 | | 0.5445 | 21.0 | 21000 | 2.0021 | 0.5302 | 0.3411 | 0.4784 | 0.4783 | 31.0095 | | 0.5355 | 22.0 | 22000 | 2.0283 | 0.5318 | 0.3432 | 0.4792 | 0.4794 | 30.2985 | | 0.5172 | 23.0 | 23000 | 2.0588 | 0.5296 | 0.3413 | 0.4775 | 0.4774 | 30.463 | | 0.4968 | 24.0 | 24000 | 2.0907 | 0.5311 | 0.3423 | 0.4781 | 0.478 | 31.0295 | | 0.4821 | 25.0 | 25000 | 2.0964 | 0.5318 | 0.3428 | 0.4792 | 0.4793 | 30.8365 | | 0.4727 | 26.0 | 26000 | 2.1195 | 0.5317 | 0.3424 | 0.4789 | 0.4788 | 30.391 | | 0.458 | 27.0 | 27000 | 2.1357 | 0.5301 | 0.3391 | 0.4761 | 0.4761 | 30.9145 | | 0.4454 | 28.0 | 28000 | 2.1648 | 0.531 | 0.3409 | 0.4774 | 0.4774 | 31.1835 | | 0.444 | 29.0 | 29000 | 2.1570 | 0.532 | 0.3418 | 0.4792 | 0.4791 | 30.596 | | 0.4349 | 30.0 | 30000 | 2.1612 | 0.5309 | 0.3406 | 0.4779 | 0.4778 | 30.6175 | ### Framework versions - Transformers 4.45.1 - Pytorch 2.2.1 - Datasets 3.0.1 - Tokenizers 0.20.0