File size: 2,796 Bytes
b67ea4f cf44004 b67ea4f cf44004 b67ea4f cf44004 b67ea4f cf44004 b67ea4f cf44004 b67ea4f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
---
license: mit
language:
- de
pipeline_tag: text-classification
metrics:
- f1
library_name: transformers
---
# PopBERT
PopBERT is a model for German-language populism detection in political speeches within the German Bundestag, based on the deepset/gbert-large model: https://huggingface.co/deepset/gbert-large
It is a multilabel model trained on a manually curated dataset of sentences from the 18th and 19th legislative periods.
In addition to capturing the foundational dimensions of populism, namely "anti-elitism" and "people-centrism," the model was also fine-tuned to identify the underlying ideological orientation as either "left-wing" or "right-wing."
# Prediction
The model outputs a Tensor of length 4.
The table connects the position of the predicted probability to its dimension.
| **Index** | **Dimension** |
|-----------|--------------------------|
| 0 | Anti-Elitism |
| 1 | People-Centrism |
| 2 | Left-Wing Host-Ideology |
| 3 | Right-Wing Host-Ideology |
# Usage Example
```python
import torch
from transformers import AutoModelForSequenceClassification
from transformers import AutoTokenizer
# load tokenizer
tokenizer = AutoTokenizer.from_pretrained("luerhard/PopBERT")
# load model
model = AutoModelForSequenceClassification.from_pretrained("luerhard/PopBERT")
# define text to be predicted
text = (
"Das ist Klassenkampf von oben, das ist Klassenkampf im Interesse von "
"Vermögenden und Besitzenden gegen die Mehrheit der Steuerzahlerinnen und "
"Steuerzahler auf dieser Erde."
)
# encode text with tokenizer
encodings = tokenizer(text, return_tensors="pt")
# predict
with torch.inference_mode():
out = model(**encodings)
# get probabilties
probs = torch.nn.functional.sigmoid(out.logits)
print(probs.detach().numpy())
```
```
[[0.8765146 0.34838045 0.983123 0.02148379]]
```
# Performance
To maximize performance, it is recommended to use the following thresholds per dimension:
```
[0.415961, 0.295400, 0.429109, 0.302714]
```
Using these thresholds, the model achieves the following performance on the test set:
| Dimension | Precision | Recall | F1 |
|---------------------|---------------|---------------|---------------|
| Anti-Elitism | 0.81 | 0.88 | 0.84 |
| People-Centrism | 0.70 | 0.73 | 0.71 |
| Left-Wing Ideology | 0.69 | 0.77 | 0.73 |
| Right-Wing Ideology | 0.68 | 0.66 | 0.67 |
| --- | --- | --- | --- |
| micro avg | 0.75 | 0.80 | 0.77 |
| macro avg | 0.72 | 0.76 | 0.74 |
|