go-emotions-plus-other-datasets-fine-tuned-distilroberta
This model is a fine-tuned version of distilbert/distilroberta-base on the these datasets:
- GoEmotions
- sem_eval_2018_task_1 (English)
- Emotion Detection from Text - Pashupati Gupta
- Emotions dataset for NLP - praveengovi
It achieves the following results on the evaluation set:
- Loss: 0.0719
- Micro Precision: 0.7358
- Micro Recall: 0.5840
- Micro F1: 0.6512
- Macro Precision: 0.5957
- Macro Recall: 0.4191
- Macro F1: 0.4670
- Weighted Precision: 0.7120
- Weighted Recall: 0.5840
- Weighted F1: 0.6286
- Hamming Loss: 0.0266
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- num_epochs: 3.0
Training results
Training Loss | Epoch | Step | Validation Loss | Micro Precision | Micro Recall | Micro F1 | Macro Precision | Macro Recall | Macro F1 | Weighted Precision | Weighted Recall | Weighted F1 | Hamming Loss |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
No log | 1.0 | 10377 | 0.0788 | 0.7494 | 0.4946 | 0.5959 | 0.5505 | 0.3191 | 0.3567 | 0.7217 | 0.4946 | 0.5559 | 0.0285 |
No log | 2.0 | 20754 | 0.0723 | 0.7354 | 0.5782 | 0.6474 | 0.6452 | 0.3792 | 0.4259 | 0.7312 | 0.5782 | 0.6154 | 0.0268 |
No log | 3.0 | 31131 | 0.0719 | 0.7358 | 0.5840 | 0.6512 | 0.5957 | 0.4191 | 0.4670 | 0.7120 | 0.5840 | 0.6286 | 0.0266 |
Test results
Threshold = 0.5
Emotion | Precision | Recall | F1-Score | Support |
---|---|---|---|---|
admiration | 0.65 | 0.69 | 0.67 | 504 |
amusement | 0.72 | 0.89 | 0.80 | 264 |
anger | 0.78 | 0.70 | 0.74 | 1585 |
annoyance | 0.51 | 0.11 | 0.18 | 320 |
approval | 0.58 | 0.31 | 0.41 | 351 |
caring | 0.50 | 0.27 | 0.35 | 135 |
confusion | 0.51 | 0.33 | 0.40 | 153 |
curiosity | 0.52 | 0.49 | 0.50 | 284 |
desire | 0.46 | 0.25 | 0.33 | 83 |
disappointment | 0.58 | 0.09 | 0.16 | 151 |
disapproval | 0.51 | 0.28 | 0.36 | 267 |
disgust | 0.72 | 0.64 | 0.68 | 1222 |
embarrassment | 0.78 | 0.19 | 0.30 | 37 |
excitement | 0.54 | 0.36 | 0.43 | 103 |
fear | 0.78 | 0.75 | 0.77 | 787 |
gratitude | 0.92 | 0.89 | 0.90 | 352 |
grief | 0.00 | 0.00 | 0.00 | 6 |
joy | 0.87 | 0.77 | 0.82 | 2298 |
love | 0.72 | 0.60 | 0.65 | 1305 |
nervousness | 0.00 | 0.00 | 0.00 | 23 |
optimism | 0.71 | 0.56 | 0.63 | 1329 |
pride | 0.00 | 0.00 | 0.00 | 16 |
realization | 0.46 | 0.11 | 0.18 | 145 |
relief | 0.59 | 0.08 | 0.14 | 160 |
remorse | 0.56 | 0.77 | 0.65 | 56 |
sadness | 0.78 | 0.67 | 0.72 | 2212 |
surprise | 0.59 | 0.28 | 0.38 | 572 |
neutral | 0.69 | 0.56 | 0.62 | 2668 |
Micro Avg | 0.74 | 0.60 | 0.66 | 17388 |
Macro Avg | 0.57 | 0.42 | 0.46 | 17388 |
Weighted Avg | 0.72 | 0.60 | 0.65 | 17388 |
Samples Avg | 0.64 | 0.61 | 0.61 | 17388 |
Framework versions
- Transformers 4.47.0
- Pytorch 2.3.1+cu121
- Datasets 2.20.0
- Tokenizers 0.21.0
- Downloads last month
- 5
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for sangkm/go-emotions-plus-other-datasets-fine-tuned-distilroberta
Base model
distilbert/distilroberta-base