metadata
license: apache-2.0
base_model: distilroberta-base
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: emotions_distilroberta
results: []
emotions_distilroberta
This model is a fine-tuned version of distilroberta-base on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.5345
- F1 Micro: 0.6750
- F1 Macro: 0.5924
- Accuracy: 0.2078
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 128
- eval_batch_size: 128
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
- mixed_precision_training: Native AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | F1 Micro | F1 Macro | Accuracy |
|---|---|---|---|---|---|---|
| 0.7583 | 0.41 | 20 | 0.6581 | 0.5780 | 0.3558 | 0.1055 |
| 0.6291 | 0.82 | 40 | 0.5893 | 0.6285 | 0.4836 | 0.1424 |
| 0.5796 | 1.22 | 60 | 0.5759 | 0.6319 | 0.5261 | 0.1618 |
| 0.5288 | 1.63 | 80 | 0.5409 | 0.6585 | 0.5526 | 0.1430 |
| 0.5111 | 2.04 | 100 | 0.5339 | 0.6681 | 0.5648 | 0.1961 |
| 0.4635 | 2.45 | 120 | 0.5291 | 0.6684 | 0.5714 | 0.1786 |
| 0.4544 | 2.86 | 140 | 0.5282 | 0.6726 | 0.5787 | 0.1618 |
| 0.4398 | 3.27 | 160 | 0.5281 | 0.6736 | 0.5833 | 0.2052 |
| 0.3948 | 3.67 | 180 | 0.5309 | 0.6650 | 0.5896 | 0.1890 |
| 0.41 | 4.08 | 200 | 0.5265 | 0.6785 | 0.5782 | 0.2168 |
| 0.3722 | 4.49 | 220 | 0.5345 | 0.6750 | 0.5924 | 0.2078 |
| 0.3617 | 4.9 | 240 | 0.5295 | 0.6769 | 0.5822 | 0.2155 |
| 0.3362 | 5.31 | 260 | 0.5358 | 0.6696 | 0.5854 | 0.1851 |
| 0.3204 | 5.71 | 280 | 0.5438 | 0.6762 | 0.5747 | 0.2097 |
| 0.3194 | 6.12 | 300 | 0.5503 | 0.6764 | 0.5768 | 0.1832 |
| 0.2921 | 6.53 | 320 | 0.5599 | 0.6734 | 0.5787 | 0.1961 |
| 0.2938 | 6.94 | 340 | 0.5532 | 0.6753 | 0.5863 | 0.1806 |
| 0.2708 | 7.35 | 360 | 0.5634 | 0.6735 | 0.5782 | 0.1922 |
| 0.2625 | 7.76 | 380 | 0.5716 | 0.6727 | 0.5756 | 0.1961 |
| 0.2565 | 8.16 | 400 | 0.5671 | 0.6739 | 0.5798 | 0.1922 |
| 0.2403 | 8.57 | 420 | 0.5816 | 0.6688 | 0.5735 | 0.1728 |
| 0.2466 | 8.98 | 440 | 0.5818 | 0.6739 | 0.5744 | 0.1871 |
| 0.2331 | 9.39 | 460 | 0.5826 | 0.6722 | 0.5762 | 0.1922 |
| 0.2233 | 9.8 | 480 | 0.5843 | 0.6738 | 0.5768 | 0.1942 |
Framework versions
- Transformers 4.39.3
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2