deberta-v3-large-sentiment
This model is a fine-tuned version of microsoft/deberta-v3-large on an tweet_eval dataset.
Model description
Test set results:
Model | Emotion | Hate | Irony | Offensive | Sentiment |
---|---|---|---|---|---|
deberta-v3-large | 86.3 | 61.3 | 87.1 | 86.4 | 73.9 |
BERTweet | 79.3 | - | 82.1 | 79.5 | 73.4 |
RoB-RT | 79.5 | 52.3 | 61.7 | 80.5 | 69.3 |
Intended uses & limitations
Classifying attributes of interest on tweeter like data.
Training and evaluation data
tweet_eval dataset.
Training procedure
Fine tuned and evaluated with run_glue.py
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 50
- num_epochs: 10.0
Training results
Training Loss | Epoch | Step | Validation Loss | Accuracy |
---|---|---|---|---|
1.0614 | 0.07 | 100 | 1.0196 | 0.4345 |
0.8601 | 0.14 | 200 | 0.7561 | 0.6460 |
0.734 | 0.21 | 300 | 0.6796 | 0.6955 |
0.6753 | 0.28 | 400 | 0.6521 | 0.7000 |
0.6408 | 0.35 | 500 | 0.6119 | 0.7440 |
0.5991 | 0.42 | 600 | 0.6034 | 0.7370 |
0.6069 | 0.49 | 700 | 0.5976 | 0.7375 |
0.6122 | 0.56 | 800 | 0.5871 | 0.7425 |
0.5908 | 0.63 | 900 | 0.5935 | 0.7445 |
0.5884 | 0.7 | 1000 | 0.5792 | 0.7520 |
0.5839 | 0.77 | 1100 | 0.5780 | 0.7555 |
0.5772 | 0.84 | 1200 | 0.5727 | 0.7570 |
0.5895 | 0.91 | 1300 | 0.5601 | 0.7550 |
0.5757 | 0.98 | 1400 | 0.5613 | 0.7525 |
0.5121 | 1.05 | 1500 | 0.5867 | 0.7600 |
0.5254 | 1.12 | 1600 | 0.5595 | 0.7630 |
0.5074 | 1.19 | 1700 | 0.5594 | 0.7585 |
0.4947 | 1.26 | 1800 | 0.5697 | 0.7575 |
0.5019 | 1.33 | 1900 | 0.5665 | 0.7580 |
0.5005 | 1.4 | 2000 | 0.5484 | 0.7655 |
0.5125 | 1.47 | 2100 | 0.5626 | 0.7605 |
0.5241 | 1.54 | 2200 | 0.5561 | 0.7560 |
0.5198 | 1.61 | 2300 | 0.5602 | 0.7600 |
0.5124 | 1.68 | 2400 | 0.5654 | 0.7490 |
0.5096 | 1.75 | 2500 | 0.5803 | 0.7515 |
0.4885 | 1.82 | 2600 | 0.5889 | 0.75 |
0.5111 | 1.89 | 2700 | 0.5508 | 0.7665 |
0.4868 | 1.96 | 2800 | 0.5621 | 0.7635 |
0.4599 | 2.04 | 2900 | 0.5995 | 0.7615 |
0.4147 | 2.11 | 3000 | 0.6202 | 0.7530 |
0.4233 | 2.18 | 3100 | 0.5875 | 0.7625 |
0.4324 | 2.25 | 3200 | 0.5794 | 0.7610 |
0.4141 | 2.32 | 3300 | 0.5902 | 0.7460 |
0.4306 | 2.39 | 3400 | 0.6053 | 0.7545 |
0.4266 | 2.46 | 3500 | 0.5979 | 0.7570 |
0.4227 | 2.53 | 3600 | 0.5920 | 0.7650 |
0.4226 | 2.6 | 3700 | 0.6166 | 0.7455 |
0.3978 | 2.67 | 3800 | 0.6126 | 0.7560 |
0.3954 | 2.74 | 3900 | 0.6152 | 0.7550 |
0.4209 | 2.81 | 4000 | 0.5980 | 0.75 |
0.3982 | 2.88 | 4100 | 0.6096 | 0.7490 |
0.4016 | 2.95 | 4200 | 0.6541 | 0.7425 |
0.3966 | 3.02 | 4300 | 0.6377 | 0.7545 |
0.3074 | 3.09 | 4400 | 0.6860 | 0.75 |
0.3551 | 3.16 | 4500 | 0.6160 | 0.7550 |
0.3323 | 3.23 | 4600 | 0.6714 | 0.7520 |
0.3171 | 3.3 | 4700 | 0.6538 | 0.7535 |
0.3403 | 3.37 | 4800 | 0.6774 | 0.7465 |
0.3396 | 3.44 | 4900 | 0.6726 | 0.7465 |
0.3259 | 3.51 | 5000 | 0.6465 | 0.7480 |
0.3392 | 3.58 | 5100 | 0.6860 | 0.7460 |
0.3251 | 3.65 | 5200 | 0.6697 | 0.7495 |
0.3253 | 3.72 | 5300 | 0.6770 | 0.7430 |
0.3455 | 3.79 | 5400 | 0.7177 | 0.7360 |
0.3323 | 3.86 | 5500 | 0.6943 | 0.7400 |
0.3335 | 3.93 | 5600 | 0.6507 | 0.7555 |
0.3368 | 4.0 | 5700 | 0.6580 | 0.7485 |
0.2479 | 4.07 | 5800 | 0.7667 | 0.7430 |
0.2613 | 4.14 | 5900 | 0.7513 | 0.7505 |
0.2557 | 4.21 | 6000 | 0.7927 | 0.7485 |
0.243 | 4.28 | 6100 | 0.7792 | 0.7450 |
0.2473 | 4.35 | 6200 | 0.8107 | 0.7355 |
0.2447 | 4.42 | 6300 | 0.7851 | 0.7370 |
0.2515 | 4.49 | 6400 | 0.7529 | 0.7465 |
0.274 | 4.56 | 6500 | 0.7390 | 0.7465 |
0.2674 | 4.63 | 6600 | 0.7658 | 0.7460 |
0.2416 | 4.7 | 6700 | 0.7915 | 0.7485 |
0.2432 | 4.77 | 6800 | 0.7989 | 0.7435 |
0.2595 | 4.84 | 6900 | 0.7850 | 0.7380 |
0.2736 | 4.91 | 7000 | 0.7577 | 0.7395 |
0.2783 | 4.98 | 7100 | 0.7650 | 0.7405 |
0.2304 | 5.05 | 7200 | 0.8542 | 0.7385 |
0.1937 | 5.12 | 7300 | 0.8390 | 0.7345 |
0.1878 | 5.19 | 7400 | 0.9150 | 0.7330 |
0.1921 | 5.26 | 7500 | 0.8792 | 0.7405 |
0.1916 | 5.33 | 7600 | 0.8892 | 0.7410 |
0.2011 | 5.4 | 7700 | 0.9012 | 0.7325 |
0.211 | 5.47 | 7800 | 0.8608 | 0.7420 |
0.2194 | 5.54 | 7900 | 0.8852 | 0.7320 |
0.205 | 5.61 | 8000 | 0.8803 | 0.7385 |
0.1981 | 5.68 | 8100 | 0.8681 | 0.7330 |
0.1908 | 5.75 | 8200 | 0.9020 | 0.7435 |
0.1942 | 5.82 | 8300 | 0.8780 | 0.7410 |
0.1958 | 5.89 | 8400 | 0.8937 | 0.7345 |
0.1883 | 5.96 | 8500 | 0.9121 | 0.7360 |
0.1819 | 6.04 | 8600 | 0.9409 | 0.7430 |
0.145 | 6.11 | 8700 | 1.1390 | 0.7265 |
0.1696 | 6.18 | 8800 | 0.9189 | 0.7430 |
0.1488 | 6.25 | 8900 | 0.9718 | 0.7400 |
0.1637 | 6.32 | 9000 | 0.9702 | 0.7450 |
0.1547 | 6.39 | 9100 | 1.0033 | 0.7410 |
0.1605 | 6.46 | 9200 | 0.9973 | 0.7355 |
0.1552 | 6.53 | 9300 | 1.0491 | 0.7290 |
0.1731 | 6.6 | 9400 | 1.0271 | 0.7335 |
0.1738 | 6.67 | 9500 | 0.9575 | 0.7430 |
0.1669 | 6.74 | 9600 | 0.9614 | 0.7350 |
0.1347 | 6.81 | 9700 | 1.0263 | 0.7365 |
0.1593 | 6.88 | 9800 | 1.0173 | 0.7360 |
0.1549 | 6.95 | 9900 | 1.0398 | 0.7350 |
0.1675 | 7.02 | 10000 | 0.9975 | 0.7380 |
0.1182 | 7.09 | 10100 | 1.1059 | 0.7350 |
0.1351 | 7.16 | 10200 | 1.0933 | 0.7400 |
0.1496 | 7.23 | 10300 | 1.0731 | 0.7355 |
0.1197 | 7.3 | 10400 | 1.1089 | 0.7360 |
0.1111 | 7.37 | 10500 | 1.1381 | 0.7405 |
0.1494 | 7.44 | 10600 | 1.0252 | 0.7425 |
0.1235 | 7.51 | 10700 | 1.0906 | 0.7360 |
0.133 | 7.58 | 10800 | 1.1796 | 0.7375 |
0.1248 | 7.65 | 10900 | 1.1332 | 0.7420 |
0.1268 | 7.72 | 11000 | 1.1304 | 0.7415 |
0.1368 | 7.79 | 11100 | 1.1345 | 0.7380 |
0.1228 | 7.86 | 11200 | 1.2018 | 0.7320 |
0.1281 | 7.93 | 11300 | 1.1884 | 0.7350 |
0.1449 | 8.0 | 11400 | 1.1571 | 0.7345 |
0.1025 | 8.07 | 11500 | 1.1538 | 0.7345 |
0.1199 | 8.14 | 11600 | 1.2113 | 0.7390 |
0.1016 | 8.21 | 11700 | 1.2882 | 0.7370 |
0.114 | 8.28 | 11800 | 1.2872 | 0.7390 |
0.1019 | 8.35 | 11900 | 1.2876 | 0.7380 |
0.1142 | 8.42 | 12000 | 1.2791 | 0.7385 |
0.1135 | 8.49 | 12100 | 1.2883 | 0.7380 |
0.1139 | 8.56 | 12200 | 1.2829 | 0.7360 |
0.1107 | 8.63 | 12300 | 1.2698 | 0.7365 |
0.1183 | 8.7 | 12400 | 1.2660 | 0.7345 |
0.1064 | 8.77 | 12500 | 1.2889 | 0.7365 |
0.0895 | 8.84 | 12600 | 1.3480 | 0.7330 |
0.1244 | 8.91 | 12700 | 1.2872 | 0.7325 |
0.1209 | 8.98 | 12800 | 1.2681 | 0.7375 |
0.1144 | 9.05 | 12900 | 1.2711 | 0.7370 |
0.1034 | 9.12 | 13000 | 1.2801 | 0.7360 |
0.113 | 9.19 | 13100 | 1.2801 | 0.7350 |
0.0994 | 9.26 | 13200 | 1.2920 | 0.7360 |
0.0966 | 9.33 | 13300 | 1.2761 | 0.7335 |
0.0939 | 9.4 | 13400 | 1.2909 | 0.7365 |
0.0975 | 9.47 | 13500 | 1.2953 | 0.7360 |
0.0842 | 9.54 | 13600 | 1.3179 | 0.7335 |
0.0871 | 9.61 | 13700 | 1.3149 | 0.7385 |
0.1162 | 9.68 | 13800 | 1.3124 | 0.7350 |
0.085 | 9.75 | 13900 | 1.3207 | 0.7355 |
0.0966 | 9.82 | 14000 | 1.3248 | 0.7335 |
0.1064 | 9.89 | 14100 | 1.3261 | 0.7335 |
0.1046 | 9.96 | 14200 | 1.3255 | 0.7360 |
Framework versions
- Transformers 4.20.0.dev0
- Pytorch 1.9.0
- Datasets 2.2.2
- Tokenizers 0.11.6
- Downloads last month
- 32
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.