|
--- |
|
license: mit |
|
language: |
|
- de |
|
pipeline_tag: text-classification |
|
metrics: |
|
- f1 |
|
library_name: transformers |
|
--- |
|
|
|
# PopBERT |
|
|
|
PopBERT is a model for German-language populism detection in political speeches within the German Bundestag, based on the deepset/gbert-large model: https://huggingface.co/deepset/gbert-large |
|
|
|
It is a multilabel model trained on a manually curated dataset of sentences from the 18th and 19th legislative periods. |
|
In addition to capturing the foundational dimensions of populism, namely "anti-elitism" and "people-centrism," the model was also fine-tuned to identify the underlying ideological orientation as either "left-wing" or "right-wing." |
|
|
|
# Prediction |
|
|
|
The model outputs a Tensor of length 4. |
|
The table connects the position of the predicted probability to its dimension. |
|
|
|
| **Index** | **Dimension** | |
|
|-----------|--------------------------| |
|
| 0 | Anti-Elitism | |
|
| 1 | People-Centrism | |
|
| 2 | Left-Wing Host-Ideology | |
|
| 3 | Right-Wing Host-Ideology | |
|
|
|
# Usage Example |
|
|
|
```python |
|
import torch |
|
from transformers import AutoModelForSequenceClassification |
|
from transformers import AutoTokenizer |
|
|
|
# load tokenizer |
|
tokenizer = AutoTokenizer.from_pretrained("luerhard/PopBERT") |
|
|
|
# load model |
|
model = AutoModelForSequenceClassification.from_pretrained("luerhard/PopBERT") |
|
|
|
# define text to be predicted |
|
text = ( |
|
"Das ist Klassenkampf von oben, das ist Klassenkampf im Interesse von " |
|
"Vermögenden und Besitzenden gegen die Mehrheit der Steuerzahlerinnen und " |
|
"Steuerzahler auf dieser Erde." |
|
) |
|
|
|
# encode text with tokenizer |
|
encodings = tokenizer(text, return_tensors="pt") |
|
|
|
# predict |
|
with torch.inference_mode(): |
|
out = model(**encodings) |
|
|
|
# get probabilties |
|
probs = torch.nn.functional.sigmoid(out.logits) |
|
print(probs.detach().numpy()) |
|
``` |
|
|
|
``` |
|
[[0.8765146 0.34838045 0.983123 0.02148379]] |
|
``` |
|
|
|
|
|
# Performance |
|
|
|
To maximize performance, it is recommended to use the following thresholds per dimension: |
|
|
|
``` |
|
[0.415961, 0.295400, 0.429109, 0.302714] |
|
``` |
|
|
|
Using these thresholds, the model achieves the following performance on the test set: |
|
|
|
| Dimension | Precision | Recall | F1 | |
|
|---------------------|---------------|---------------|---------------| |
|
| Anti-Elitism | 0.81 | 0.88 | 0.84 | |
|
| People-Centrism | 0.70 | 0.73 | 0.71 | |
|
| Left-Wing Ideology | 0.69 | 0.77 | 0.73 | |
|
| Right-Wing Ideology | 0.68 | 0.66 | 0.67 | |
|
| --- | --- | --- | --- | |
|
| micro avg | 0.75 | 0.80 | 0.77 | |
|
| macro avg | 0.72 | 0.76 | 0.74 | |
|
|
