hyenadna-medium-160k-seqlen-hf_ft_BioS2_1kbpHG19_DHSs_H3K27AC
This model is a fine-tuned version of LongSafari/hyenadna-medium-160k-seqlen-hf on the None dataset. It achieves the following results on the evaluation set:
- Loss: 0.4893
- F1 Score: 0.8260
- Precision: 0.7676
- Recall: 0.8941
- Accuracy: 0.8037
- Auc: 0.8850
- Prc: 0.8777
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
- mixed_precision_training: Native AMP
Training results
Training Loss | Epoch | Step | Validation Loss | F1 Score | Precision | Recall | Accuracy | Auc | Prc |
---|---|---|---|---|---|---|---|---|---|
0.554 | 0.1678 | 500 | 0.5523 | 0.6784 | 0.7894 | 0.5948 | 0.7060 | 0.8144 | 0.7936 |
0.5267 | 0.3356 | 1000 | 0.4857 | 0.7992 | 0.7245 | 0.8912 | 0.7666 | 0.8409 | 0.8211 |
0.4995 | 0.5034 | 1500 | 0.5247 | 0.7956 | 0.6810 | 0.9565 | 0.7438 | 0.8443 | 0.8191 |
0.4755 | 0.6711 | 2000 | 0.4548 | 0.8058 | 0.7793 | 0.8342 | 0.7904 | 0.8619 | 0.8468 |
0.4578 | 0.8389 | 2500 | 0.4798 | 0.7471 | 0.8350 | 0.6759 | 0.7614 | 0.8664 | 0.8614 |
0.4543 | 1.0067 | 3000 | 0.4430 | 0.8082 | 0.7822 | 0.8359 | 0.7931 | 0.8743 | 0.8660 |
0.44 | 1.1745 | 3500 | 0.4306 | 0.8168 | 0.7751 | 0.8632 | 0.7982 | 0.8786 | 0.8726 |
0.4235 | 1.3423 | 4000 | 0.4809 | 0.8190 | 0.7707 | 0.8738 | 0.7987 | 0.8783 | 0.8689 |
0.4178 | 1.5101 | 4500 | 0.4363 | 0.8117 | 0.8076 | 0.8159 | 0.8027 | 0.8823 | 0.8760 |
0.4178 | 1.6779 | 5000 | 0.4392 | 0.8203 | 0.7734 | 0.8732 | 0.8005 | 0.8824 | 0.8767 |
0.4178 | 1.8456 | 5500 | 0.4148 | 0.8200 | 0.8077 | 0.8326 | 0.8094 | 0.8899 | 0.8853 |
0.4134 | 2.0134 | 6000 | 0.4275 | 0.8148 | 0.8169 | 0.8127 | 0.8074 | 0.8882 | 0.8821 |
0.3808 | 2.1812 | 6500 | 0.4376 | 0.8149 | 0.7910 | 0.8404 | 0.8010 | 0.8847 | 0.8811 |
0.3854 | 2.3490 | 7000 | 0.4326 | 0.8241 | 0.7560 | 0.9057 | 0.7985 | 0.8885 | 0.8845 |
0.374 | 2.5168 | 7500 | 0.4445 | 0.8198 | 0.8016 | 0.8388 | 0.8077 | 0.8870 | 0.8828 |
0.3844 | 2.6846 | 8000 | 0.4454 | 0.8251 | 0.7468 | 0.9218 | 0.7963 | 0.8894 | 0.8838 |
0.3817 | 2.8523 | 8500 | 0.4368 | 0.8115 | 0.8159 | 0.8072 | 0.8045 | 0.8880 | 0.8833 |
0.3838 | 3.0201 | 9000 | 0.4934 | 0.7985 | 0.8372 | 0.7631 | 0.7992 | 0.8887 | 0.8811 |
0.3444 | 3.1879 | 9500 | 0.4439 | 0.8257 | 0.7778 | 0.8799 | 0.8064 | 0.8856 | 0.8803 |
0.3336 | 3.3557 | 10000 | 0.4541 | 0.8128 | 0.8179 | 0.8079 | 0.8060 | 0.8888 | 0.8856 |
0.3524 | 3.5235 | 10500 | 0.4765 | 0.8266 | 0.7449 | 0.9285 | 0.7970 | 0.8879 | 0.8811 |
0.3532 | 3.6913 | 11000 | 0.4351 | 0.8259 | 0.7786 | 0.8793 | 0.8067 | 0.8848 | 0.8791 |
0.3342 | 3.8591 | 11500 | 0.4457 | 0.8227 | 0.8040 | 0.8423 | 0.8107 | 0.8878 | 0.8826 |
0.342 | 4.0268 | 12000 | 0.4544 | 0.8153 | 0.7777 | 0.8568 | 0.7977 | 0.8805 | 0.8792 |
0.2962 | 4.1946 | 12500 | 0.5764 | 0.7800 | 0.8422 | 0.7264 | 0.7864 | 0.8819 | 0.8804 |
0.3026 | 4.3624 | 13000 | 0.4893 | 0.8260 | 0.7676 | 0.8941 | 0.8037 | 0.8850 | 0.8777 |
Framework versions
- Transformers 4.42.3
- Pytorch 2.3.0+cu121
- Datasets 2.18.0
- Tokenizers 0.19.0
- Downloads last month
- 8
Model tree for tanoManzo/hyenadna-medium-160k-seqlen-hf_ft_BioS2_1kbpHG19_DHSs_H3K27AC
Base model
LongSafari/hyenadna-medium-160k-seqlen-hf