---
license: apache-2.0
base_model: microsoft/beit-base-patch16-224
tags:
- generated_from_trainer
datasets:
- imagefolder
metrics:
- accuracy
model-index:
- name: beit-base-patch16-224-RD
  results:
  - task:
      name: Image Classification
      type: image-classification
    dataset:
      name: imagefolder
      type: imagefolder
      config: default
      split: validation
      args: default
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.8672727272727273
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# beit-base-patch16-224-RD

This model is a fine-tuned version of [microsoft/beit-base-patch16-224](https://huggingface.co/microsoft/beit-base-patch16-224) on the imagefolder dataset.
It achieves the following results on the evaluation set:
- Loss: 0.3771
- Accuracy: 0.8673

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 64
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 40

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 1.4986        | 0.99  | 40   | 1.4512          | 0.4945   |
| 1.0553        | 1.99  | 80   | 0.9355          | 0.7473   |
| 0.7972        | 2.98  | 120  | 0.7250          | 0.7436   |
| 0.7156        | 4.0   | 161  | 0.5845          | 0.7582   |
| 0.6723        | 4.99  | 201  | 0.5509          | 0.8036   |
| 0.5942        | 5.99  | 241  | 0.5018          | 0.8218   |
| 0.6223        | 6.98  | 281  | 0.4993          | 0.8218   |
| 0.5731        | 8.0   | 322  | 0.4590          | 0.8291   |
| 0.5583        | 8.99  | 362  | 0.4878          | 0.8      |
| 0.5784        | 9.99  | 402  | 0.4485          | 0.8455   |
| 0.4968        | 10.98 | 442  | 0.4305          | 0.8345   |
| 0.5324        | 12.0  | 483  | 0.4737          | 0.8345   |
| 0.4629        | 12.99 | 523  | 0.4253          | 0.8436   |
| 0.4398        | 13.99 | 563  | 0.4184          | 0.8473   |
| 0.4575        | 14.98 | 603  | 0.3929          | 0.8564   |
| 0.4554        | 16.0  | 644  | 0.4282          | 0.8491   |
| 0.4646        | 16.99 | 684  | 0.4363          | 0.8236   |
| 0.4535        | 17.99 | 724  | 0.4337          | 0.8455   |
| 0.3823        | 18.98 | 764  | 0.3771          | 0.8673   |
| 0.4584        | 20.0  | 805  | 0.3966          | 0.8564   |
| 0.4103        | 20.99 | 845  | 0.4001          | 0.8491   |
| 0.3659        | 21.99 | 885  | 0.3948          | 0.8582   |
| 0.3241        | 22.98 | 925  | 0.4007          | 0.8582   |
| 0.3575        | 24.0  | 966  | 0.4328          | 0.8327   |
| 0.3411        | 24.99 | 1006 | 0.3990          | 0.8564   |
| 0.3829        | 25.99 | 1046 | 0.4011          | 0.8636   |
| 0.2855        | 26.98 | 1086 | 0.3859          | 0.8655   |
| 0.254         | 28.0  | 1127 | 0.4196          | 0.8673   |
| 0.2937        | 28.99 | 1167 | 0.4340          | 0.8618   |
| 0.258         | 29.99 | 1207 | 0.4387          | 0.8509   |
| 0.2735        | 30.98 | 1247 | 0.4097          | 0.8655   |
| 0.2674        | 32.0  | 1288 | 0.4183          | 0.8527   |
| 0.2547        | 32.99 | 1328 | 0.4217          | 0.8636   |
| 0.2109        | 33.99 | 1368 | 0.4240          | 0.8527   |
| 0.2248        | 34.98 | 1408 | 0.4250          | 0.86     |
| 0.2397        | 36.0  | 1449 | 0.4431          | 0.8582   |
| 0.1823        | 36.99 | 1489 | 0.4442          | 0.8582   |
| 0.1834        | 37.99 | 1529 | 0.4362          | 0.8618   |
| 0.1864        | 38.98 | 1569 | 0.4338          | 0.8545   |
| 0.1779        | 39.75 | 1600 | 0.4332          | 0.8582   |


### Framework versions

- Transformers 4.36.2
- Pytorch 2.1.2+cu118
- Datasets 2.16.1
- Tokenizers 0.15.0