Commit
·
5e3ec76
1
Parent(s):
d52e6c0
Upload README.md
Browse files
README.md
ADDED
@@ -0,0 +1,134 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
language:
|
3 |
+
- en
|
4 |
+
thumbnail: https://avatars3.githubusercontent.com/u/32437151?s=460&u=4ec59abc8d21d5feea3dab323d23a5860e6996a4&v=4
|
5 |
+
tags:
|
6 |
+
- text-classification
|
7 |
+
- emotion
|
8 |
+
- pytorch
|
9 |
+
license: apache-2.0
|
10 |
+
datasets:
|
11 |
+
- emotion
|
12 |
+
metrics:
|
13 |
+
- Accuracy, F1 Score
|
14 |
+
model-index:
|
15 |
+
- name: bhadresh-savani/distilbert-base-uncased-emotion
|
16 |
+
results:
|
17 |
+
- task:
|
18 |
+
type: text-classification
|
19 |
+
name: Text Classification
|
20 |
+
dataset:
|
21 |
+
name: emotion
|
22 |
+
type: emotion
|
23 |
+
config: default
|
24 |
+
split: test
|
25 |
+
metrics:
|
26 |
+
- name: Accuracy
|
27 |
+
type: accuracy
|
28 |
+
value: 0.927
|
29 |
+
verified: true
|
30 |
+
- name: Precision Macro
|
31 |
+
type: precision
|
32 |
+
value: 0.8880230732280744
|
33 |
+
verified: true
|
34 |
+
- name: Precision Micro
|
35 |
+
type: precision
|
36 |
+
value: 0.927
|
37 |
+
verified: true
|
38 |
+
- name: Precision Weighted
|
39 |
+
type: precision
|
40 |
+
value: 0.9272902840835793
|
41 |
+
verified: true
|
42 |
+
- name: Recall Macro
|
43 |
+
type: recall
|
44 |
+
value: 0.8790126653780703
|
45 |
+
verified: true
|
46 |
+
- name: Recall Micro
|
47 |
+
type: recall
|
48 |
+
value: 0.927
|
49 |
+
verified: true
|
50 |
+
- name: Recall Weighted
|
51 |
+
type: recall
|
52 |
+
value: 0.927
|
53 |
+
verified: true
|
54 |
+
- name: F1 Macro
|
55 |
+
type: f1
|
56 |
+
value: 0.8825061528287809
|
57 |
+
verified: true
|
58 |
+
- name: F1 Micro
|
59 |
+
type: f1
|
60 |
+
value: 0.927
|
61 |
+
verified: true
|
62 |
+
- name: F1 Weighted
|
63 |
+
type: f1
|
64 |
+
value: 0.926876082854655
|
65 |
+
verified: true
|
66 |
+
- name: loss
|
67 |
+
type: loss
|
68 |
+
value: 0.17403268814086914
|
69 |
+
verified: true
|
70 |
+
---
|
71 |
+
# Distilbert-base-uncased-emotion
|
72 |
+
|
73 |
+
## Model description:
|
74 |
+
[Distilbert](https://arxiv.org/abs/1910.01108) is created with knowledge distillation during the pre-training phase which reduces the size of a BERT model by 40%, while retaining 97% of its language understanding. It's smaller, faster than Bert and any other Bert-based model.
|
75 |
+
|
76 |
+
[Distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) finetuned on the emotion dataset using HuggingFace Trainer with below Hyperparameters
|
77 |
+
```
|
78 |
+
learning rate 2e-5,
|
79 |
+
batch size 64,
|
80 |
+
num_train_epochs=8,
|
81 |
+
```
|
82 |
+
|
83 |
+
## Model Performance Comparision on Emotion Dataset from Twitter:
|
84 |
+
|
85 |
+
| Model | Accuracy | F1 Score | Test Sample per Second |
|
86 |
+
| --- | --- | --- | --- |
|
87 |
+
| [Distilbert-base-uncased-emotion](https://huggingface.co/bhadresh-savani/distilbert-base-uncased-emotion) | 93.8 | 93.79 | 398.69 |
|
88 |
+
| [Bert-base-uncased-emotion](https://huggingface.co/bhadresh-savani/bert-base-uncased-emotion) | 94.05 | 94.06 | 190.152 |
|
89 |
+
| [Roberta-base-emotion](https://huggingface.co/bhadresh-savani/roberta-base-emotion) | 93.95 | 93.97| 195.639 |
|
90 |
+
| [Albert-base-v2-emotion](https://huggingface.co/bhadresh-savani/albert-base-v2-emotion) | 93.6 | 93.65 | 182.794 |
|
91 |
+
|
92 |
+
## How to Use the model:
|
93 |
+
```python
|
94 |
+
from transformers import pipeline
|
95 |
+
classifier = pipeline("text-classification",model='bhadresh-savani/distilbert-base-uncased-emotion', return_all_scores=True)
|
96 |
+
prediction = classifier("I love using transformers. The best part is wide range of support and its easy to use", )
|
97 |
+
print(prediction)
|
98 |
+
|
99 |
+
"""
|
100 |
+
Output:
|
101 |
+
[[
|
102 |
+
{'label': 'sadness', 'score': 0.0006792712374590337},
|
103 |
+
{'label': 'joy', 'score': 0.9959300756454468},
|
104 |
+
{'label': 'love', 'score': 0.0009452480007894337},
|
105 |
+
{'label': 'anger', 'score': 0.0018055217806249857},
|
106 |
+
{'label': 'fear', 'score': 0.00041110432357527316},
|
107 |
+
{'label': 'surprise', 'score': 0.0002288572577526793}
|
108 |
+
]]
|
109 |
+
"""
|
110 |
+
```
|
111 |
+
|
112 |
+
## Dataset:
|
113 |
+
[Twitter-Sentiment-Analysis](https://huggingface.co/nlp/viewer/?dataset=emotion).
|
114 |
+
|
115 |
+
## Training procedure
|
116 |
+
[Colab Notebook](https://github.com/bhadreshpsavani/ExploringSentimentalAnalysis/blob/main/SentimentalAnalysisWithDistilbert.ipynb)
|
117 |
+
|
118 |
+
## Eval results
|
119 |
+
```json
|
120 |
+
{
|
121 |
+
'test_accuracy': 0.938,
|
122 |
+
'test_f1': 0.937932884041714,
|
123 |
+
'test_loss': 0.1472451239824295,
|
124 |
+
'test_mem_cpu_alloc_delta': 0,
|
125 |
+
'test_mem_cpu_peaked_delta': 0,
|
126 |
+
'test_mem_gpu_alloc_delta': 0,
|
127 |
+
'test_mem_gpu_peaked_delta': 163454464,
|
128 |
+
'test_runtime': 5.0164,
|
129 |
+
'test_samples_per_second': 398.69
|
130 |
+
}
|
131 |
+
```
|
132 |
+
|
133 |
+
## Reference:
|
134 |
+
* [Natural Language Processing with Transformer By Lewis Tunstall, Leandro von Werra, Thomas Wolf](https://learning.oreilly.com/library/view/natural-language-processing/9781098103231/)
|