Commit
·
c3024c1
1
Parent(s):
c0fead2
Update README.md
Browse files
README.md
CHANGED
@@ -64,10 +64,14 @@ Please consult the original DeBERTa paper and the papers for the different datas
|
|
64 |
|
65 |
## Metrics
|
66 |
|
67 |
-
Balanced accuracy metrics on all datasets.
|
68 |
-
To calculate this metrics, 28 different models were trained, each with one dataset held out from training to simulate a zeroshot setup.
|
69 |
`deberta-v3-large-zeroshot-v1.1-all-33` was trained on all datasets, with only maximum 500 texts per class to avoid overfitting.
|
70 |
-
|
|
|
|
|
|
|
|
|
|
|
71 |
|
72 |
|
73 |
| | deberta-v3-large-mnli-fever-anli-ling-wanli-binary | deberta-v3-large-zeroshot-v1.1-heldout | deberta-v3-large-zeroshot-v1.1-all-33 |
|
|
|
64 |
|
65 |
## Metrics
|
66 |
|
67 |
+
Balanced accuracy metrics on all datasets.
|
|
|
68 |
`deberta-v3-large-zeroshot-v1.1-all-33` was trained on all datasets, with only maximum 500 texts per class to avoid overfitting.
|
69 |
+
The metrics on these datasets are therefore not strictly zeroshot, as the model has seen some data for each task.
|
70 |
+
`deberta-v3-large-zeroshot-v1.1-heldout` indicates zeroshot performance on the respective dataset.
|
71 |
+
To calculate these zeroshot metrics, 28 models were trained, each with one dataset held out from training to simulate a zeroshot setup.
|
72 |
+
The other setup (hyperparameters etc.) remained the same.
|
73 |
+
|
74 |
+
![figure_large_v1.1](https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/results/fig_large_v1.1.png)
|
75 |
|
76 |
|
77 |
| | deberta-v3-large-mnli-fever-anli-ling-wanli-binary | deberta-v3-large-zeroshot-v1.1-heldout | deberta-v3-large-zeroshot-v1.1-all-33 |
|