esm2_t12_35M_UR50D-finetuned-AMP_Classification_HighAccuracy

This model is a fine-tuned version of facebook/esm2_t12_35M_UR50D on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0059
• Train Accuracy: 0.9981
• Validation Loss: 0.0985
• Validation Accuracy: 0.9769
• Epoch: 9

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.0}
• training_precision: float32

Training results

Train Loss	Train Accuracy	Validation Loss	Validation Accuracy	Epoch
0.1093	0.9602	0.0862	0.9702	0
0.0626	0.9763	0.0717	0.9723	1
0.0372	0.9858	0.0976	0.9713	2
0.0204	0.9927	0.0913	0.9771	3
0.0121	0.9964	0.0953	0.9769	4
0.0109	0.9963	0.1082	0.9752	5
0.0085	0.9970	0.1057	0.9728	6
0.0063	0.9981	0.1145	0.9736	7
0.0051	0.9984	0.1181	0.9755	8
0.0059	0.9981	0.0985	0.9769	9

Framework versions

• Transformers 4.40.1
• TensorFlow 2.15.0
• Datasets 2.19.1
• Tokenizers 0.19.1