codet5-small-ft-v3

This model is a fine-tuned version of Salesforce/codet5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.3637
• Rouge1: 43.6813
• Rouge2: 23.5511
• Rougel: 41.3344
• Rougelsum: 41.3133
• Gen Len: 15.4815

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	5	4.0679	33.211	16.5431	32.2171	32.5082	17.1852
No log	2.0	10	2.9674	35.4549	17.5051	34.2717	34.5858	16.7407
No log	3.0	15	2.4874	40.3342	21.4844	39.1385	39.3387	16.4074
No log	4.0	20	2.3637	43.6813	23.5511	41.3344	41.3133	15.4815

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0