Edit model card

esm2_t130_150M-lora-classifier_2024-04-26_10-08-51

This model is a fine-tuned version of facebook/esm2_t30_150M_UR50D on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4537
  • Accuracy: 0.8984

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0008701568055793088
  • train_batch_size: 28
  • eval_batch_size: 28
  • seed: 8893
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
0.6764 1.0 55 0.6794 0.5820
0.5521 2.0 110 0.6192 0.6777
0.5409 3.0 165 0.5147 0.7383
0.5518 4.0 220 0.3518 0.8672
0.1386 5.0 275 0.3596 0.8574
0.303 6.0 330 0.4030 0.8359
0.1962 7.0 385 0.3143 0.8848
0.1501 8.0 440 0.3232 0.8652
0.2994 9.0 495 0.3014 0.8770
0.0914 10.0 550 0.2980 0.8887
0.2108 11.0 605 0.2854 0.8770
0.2896 12.0 660 0.3684 0.8691
0.0818 13.0 715 0.3349 0.8828
0.3152 14.0 770 0.3530 0.8848
0.0554 15.0 825 0.3371 0.8887
0.1928 16.0 880 0.3347 0.875
0.2658 17.0 935 0.3765 0.8867
0.4242 18.0 990 0.4166 0.8945
0.0964 19.0 1045 0.3400 0.8945
0.0375 20.0 1100 0.3581 0.9004
0.1781 21.0 1155 0.3816 0.8848
0.1563 22.0 1210 0.3940 0.8867
0.017 23.0 1265 0.4098 0.8926
0.1866 24.0 1320 0.4710 0.8770
0.0632 25.0 1375 0.4541 0.8828
0.1501 26.0 1430 0.4645 0.8828
0.109 27.0 1485 0.4434 0.8926
0.0353 28.0 1540 0.4264 0.8984
0.4502 29.0 1595 0.4479 0.8984
0.0341 30.0 1650 0.4537 0.8984

Framework versions

  • PEFT 0.10.0
  • Transformers 4.39.3
  • Pytorch 2.2.1
  • Datasets 2.16.1
  • Tokenizers 0.15.2
Downloads last month
0
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Model tree for wcvz/esm2_t130_150M-lora-classifier_2024-04-26_10-08-51

Adapter
(5)
this model