ke_t5_base_aihub

This model is a fine-tuned version of KETI-AIR/ke-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: nan
• Rouge1: 0.0
• Rouge2: 0.0
• Rougel: 0.0
• Rougelsum: 0.0
• Gen Len: 0.0

Model description

KE-T5 is a pretrained-model of t5 text-to-text transfer transformers using the Korean and English corpus developed by KETI (한국전자연구원). The vocabulary used by KE-T5 consists of 64,000 sub-word tokens and was created using Google's sentencepiece. The Sentencepiece model was trained to cover 99.95% of a 30GB corpus with an approximate 7:3 mix of Korean and English.

Intended uses & limitations

This is an excersize for ke-t5 summarization finetuning using pre-trained ke-t5-base using the data from aihub.

Training and evaluation data

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.0	1.0	743	nan	0.0	0.0	0.0	0.0	0.0
0.0	2.0	1486	nan	0.0	0.0	0.0	0.0	0.0
0.0	3.0	2229	nan	0.0	0.0	0.0	0.0	0.0
0.0	4.0	2972	nan	0.0	0.0	0.0	0.0	0.0

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.4
• Tokenizers 0.13.3