MoritzLaurer HF staff commited on
Commit
c3024c1
·
1 Parent(s): c0fead2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -3
README.md CHANGED
@@ -64,10 +64,14 @@ Please consult the original DeBERTa paper and the papers for the different datas
64
 
65
  ## Metrics
66
 
67
- Balanced accuracy metrics on all datasets. `deberta-v3-large-zeroshot-v1.1-heldout` indicates zeroshot performance on the respective dataset.
68
- To calculate this metrics, 28 different models were trained, each with one dataset held out from training to simulate a zeroshot setup.
69
  `deberta-v3-large-zeroshot-v1.1-all-33` was trained on all datasets, with only maximum 500 texts per class to avoid overfitting.
70
- (The metrics in the last column are therefore not strictly zeroshot.)
 
 
 
 
 
71
 
72
 
73
  | | deberta-v3-large-mnli-fever-anli-ling-wanli-binary | deberta-v3-large-zeroshot-v1.1-heldout | deberta-v3-large-zeroshot-v1.1-all-33 |
 
64
 
65
  ## Metrics
66
 
67
+ Balanced accuracy metrics on all datasets.
 
68
  `deberta-v3-large-zeroshot-v1.1-all-33` was trained on all datasets, with only maximum 500 texts per class to avoid overfitting.
69
+ The metrics on these datasets are therefore not strictly zeroshot, as the model has seen some data for each task.
70
+ `deberta-v3-large-zeroshot-v1.1-heldout` indicates zeroshot performance on the respective dataset.
71
+ To calculate these zeroshot metrics, 28 models were trained, each with one dataset held out from training to simulate a zeroshot setup.
72
+ The other setup (hyperparameters etc.) remained the same.
73
+
74
+ ![figure_large_v1.1](https://github.com/MoritzLaurer/zeroshot-classifier/blob/main/results/fig_large_v1.1.png)
75
 
76
 
77
  | | deberta-v3-large-mnli-fever-anli-ling-wanli-binary | deberta-v3-large-zeroshot-v1.1-heldout | deberta-v3-large-zeroshot-v1.1-all-33 |