esm2_t30_150M_UR50D-finetuned-AMP_Classification

This model is a fine-tuned version of facebook/esm2_t30_150M_UR50D on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.0433
• Train Accuracy: 0.9871
• Validation Loss: 0.7702
• Validation Accuracy: 0.8014
• Epoch: 19

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.0}
• training_precision: float32

Training results

Train Loss	Train Accuracy	Validation Loss	Validation Accuracy	Epoch
0.6498	0.6047	0.6345	0.6288	0
0.5714	0.6877	0.5871	0.6882	1
0.3898	0.8198	0.5698	0.7242	2
0.2481	0.8921	0.5758	0.7696	3
0.1838	0.9248	0.6483	0.7730	4
0.1475	0.9390	0.6187	0.7904	5
0.1147	0.9541	0.6663	0.8007	6
0.0948	0.9618	0.7591	0.7819	7
0.0800	0.9701	0.7534	0.7959	8
0.0709	0.9739	0.8595	0.7810	9
0.0629	0.9767	0.8192	0.7907	10
0.0578	0.9792	0.8855	0.7946	11
0.0532	0.9814	0.9993	0.7762	12
0.0586	0.9801	0.9058	0.7761	13
0.0534	0.9816	0.8338	0.7786	14
0.0508	0.9824	0.7899	0.8033	15
0.0472	0.9840	0.9000	0.7800	16
0.0441	0.9851	0.8732	0.7911	17
0.0486	0.9846	0.8166	0.8088	18
0.0433	0.9871	0.7702	0.8014	19

Framework versions

• Transformers 4.40.1
• TensorFlow 2.15.0
• Datasets 2.19.1
• Tokenizers 0.19.1