metadata
license: apache-2.0
base_model: microsoft/beit-base-patch16-224
tags:
- generated_from_trainer
datasets:
- imagefolder
metrics:
- accuracy
model-index:
- name: BEiT-DMAE-13XDA-REVAL-80-32
results:
- task:
name: Image Classification
type: image-classification
dataset:
name: imagefolder
type: imagefolder
config: default
split: validation
args: default
metrics:
- name: Accuracy
type: accuracy
value: 0.8478260869565217
BEiT-DMAE-13XDA-REVAL-80-32
This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:
- Loss: 0.8105
- Accuracy: 0.8478
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 3.5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 80
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 1.5474 | 1.0 | 60 | 1.2945 | 0.4565 |
| 1.3959 | 1.99 | 120 | 1.2745 | 0.4565 |
| 1.0517 | 2.99 | 180 | 0.9632 | 0.6087 |
| 0.7273 | 4.0 | 241 | 0.7709 | 0.6957 |
| 0.5246 | 5.0 | 301 | 0.7217 | 0.7391 |
| 0.3645 | 5.99 | 361 | 0.7142 | 0.8043 |
| 0.2211 | 6.99 | 421 | 0.6436 | 0.8043 |
| 0.266 | 8.0 | 482 | 1.1316 | 0.6087 |
| 0.1235 | 9.0 | 542 | 0.9257 | 0.7826 |
| 0.1613 | 9.99 | 602 | 0.8527 | 0.7826 |
| 0.0946 | 10.99 | 662 | 0.8274 | 0.8043 |
| 0.1392 | 12.0 | 723 | 0.8312 | 0.7609 |
| 0.1028 | 13.0 | 783 | 1.1959 | 0.7609 |
| 0.1072 | 13.99 | 843 | 1.0017 | 0.7391 |
| 0.0888 | 14.99 | 903 | 0.9214 | 0.8043 |
| 0.0951 | 16.0 | 964 | 0.9156 | 0.7609 |
| 0.0714 | 17.0 | 1024 | 1.3116 | 0.6957 |
| 0.0804 | 17.99 | 1084 | 1.1107 | 0.7826 |
| 0.08 | 18.99 | 1144 | 0.8105 | 0.8478 |
| 0.1619 | 20.0 | 1205 | 0.7581 | 0.8261 |
| 0.084 | 21.0 | 1265 | 1.0210 | 0.8261 |
| 0.072 | 21.99 | 1325 | 1.3092 | 0.7609 |
| 0.0303 | 22.99 | 1385 | 1.3367 | 0.7826 |
| 0.0228 | 24.0 | 1446 | 1.0277 | 0.8261 |
| 0.0755 | 25.0 | 1506 | 0.9436 | 0.8261 |
| 0.0756 | 25.99 | 1566 | 1.1588 | 0.7609 |
| 0.0875 | 26.99 | 1626 | 1.3280 | 0.7174 |
| 0.0771 | 28.0 | 1687 | 1.8558 | 0.6739 |
| 0.0467 | 29.0 | 1747 | 1.6476 | 0.7391 |
| 0.0382 | 29.99 | 1807 | 0.9374 | 0.8478 |
| 0.0511 | 30.99 | 1867 | 1.0847 | 0.8043 |
| 0.0161 | 32.0 | 1928 | 1.2028 | 0.7826 |
| 0.0301 | 33.0 | 1988 | 1.2971 | 0.7391 |
| 0.0443 | 33.99 | 2048 | 1.3993 | 0.7174 |
| 0.0782 | 34.99 | 2108 | 1.3359 | 0.8043 |
| 0.0287 | 36.0 | 2169 | 1.3011 | 0.7826 |
| 0.0347 | 37.0 | 2229 | 1.2450 | 0.7826 |
| 0.0538 | 37.99 | 2289 | 1.8216 | 0.7609 |
| 0.027 | 38.99 | 2349 | 1.1701 | 0.8043 |
| 0.038 | 40.0 | 2410 | 1.1025 | 0.8043 |
| 0.0244 | 41.0 | 2470 | 1.2912 | 0.7609 |
| 0.0122 | 41.99 | 2530 | 1.5699 | 0.7609 |
| 0.023 | 42.99 | 2590 | 1.5114 | 0.7826 |
| 0.0297 | 44.0 | 2651 | 1.2189 | 0.8478 |
| 0.0284 | 45.0 | 2711 | 1.3997 | 0.7826 |
| 0.0203 | 45.99 | 2771 | 1.4792 | 0.8043 |
| 0.03 | 46.99 | 2831 | 1.7487 | 0.7174 |
| 0.025 | 48.0 | 2892 | 1.6605 | 0.7609 |
| 0.0134 | 49.0 | 2952 | 1.4106 | 0.7826 |
| 0.026 | 49.99 | 3012 | 1.2972 | 0.7609 |
| 0.0507 | 50.99 | 3072 | 1.3303 | 0.7826 |
| 0.0394 | 52.0 | 3133 | 1.1954 | 0.8478 |
| 0.0271 | 53.0 | 3193 | 1.3125 | 0.8261 |
| 0.0115 | 53.99 | 3253 | 1.3444 | 0.8478 |
| 0.0138 | 54.99 | 3313 | 1.4689 | 0.8261 |
| 0.0184 | 56.0 | 3374 | 1.4959 | 0.8261 |
| 0.0163 | 57.0 | 3434 | 1.3490 | 0.7826 |
| 0.0112 | 57.99 | 3494 | 1.4749 | 0.7826 |
| 0.0185 | 58.99 | 3554 | 1.5823 | 0.7826 |
| 0.031 | 60.0 | 3615 | 1.5190 | 0.7826 |
| 0.0161 | 61.0 | 3675 | 1.5476 | 0.8043 |
| 0.0146 | 61.99 | 3735 | 1.3930 | 0.7826 |
| 0.005 | 62.99 | 3795 | 1.5454 | 0.8043 |
| 0.0093 | 64.0 | 3856 | 1.5959 | 0.7826 |
| 0.0224 | 65.0 | 3916 | 1.4554 | 0.8043 |
| 0.0154 | 65.99 | 3976 | 1.5327 | 0.8261 |
| 0.0116 | 66.99 | 4036 | 1.6030 | 0.8043 |
| 0.0037 | 68.0 | 4097 | 1.5046 | 0.8261 |
| 0.0023 | 69.0 | 4157 | 1.5222 | 0.8261 |
| 0.0068 | 69.99 | 4217 | 1.4339 | 0.8261 |
| 0.0342 | 70.99 | 4277 | 1.6964 | 0.8043 |
| 0.0077 | 72.0 | 4338 | 1.6102 | 0.8043 |
| 0.0043 | 73.0 | 4398 | 1.6687 | 0.8043 |
| 0.0131 | 73.99 | 4458 | 1.6847 | 0.8043 |
| 0.0031 | 74.99 | 4518 | 1.7195 | 0.8043 |
| 0.0087 | 76.0 | 4579 | 1.7209 | 0.7826 |
| 0.0219 | 77.0 | 4639 | 1.6715 | 0.8043 |
| 0.0229 | 77.99 | 4699 | 1.6823 | 0.8043 |
| 0.008 | 78.99 | 4759 | 1.6751 | 0.8043 |
| 0.0051 | 79.67 | 4800 | 1.6758 | 0.8043 |
Framework versions
- Transformers 4.36.2
- Pytorch 2.1.2+cu118
- Datasets 2.16.1
- Tokenizers 0.15.0