Update README.md

README.md

@@ -1,72 +1,90 @@
 ---
-license: mit
-base_model: microsoft/deberta-v3-large
+language:
+- en
 tags:
-- generated_from_trainer
-metrics:
-- accuracy
-model-index:
-- name: deberta-v3-large-zeroshot-v1.1-none
-  results: []
+- text-classification
+- zero-shot-classification
+pipeline_tag: zero-shot-classification
+library_name: transformers
+license: mit
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
-# deberta-v3-large-zeroshot-v1.1-none
-
-This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on an unknown dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.2186
-- F1 Macro: 0.6981
-- F1 Micro: 0.7601
-- Accuracy Balanced: 0.7396
-- Accuracy: 0.7601
-- Precision Macro: 0.7117
-- Recall Macro: 0.7396
-- Precision Micro: 0.7601
-- Recall Micro: 0.7601
-
+# deberta-v3-large-zeroshot-v1.1-all-33
 ## Model description
-
-More information needed
-
-## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
-
-## Training procedure
-
-### Training hyperparameters
-
-The following hyperparameters were used during training:
-- learning_rate: 9e-06
-- train_batch_size: 16
-- eval_batch_size: 64
-- seed: 42
-- gradient_accumulation_steps: 2
-- total_train_batch_size: 32
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_ratio: 0.06
-- num_epochs: 3
-
-### Training results
-
-| Training Loss | Epoch | Step  | Validation Loss | F1 Macro | F1 Micro | Accuracy Balanced | Accuracy | Precision Macro | Recall Macro | Precision Micro | Recall Micro |
-|:-------------:|:-----:|:-----:|:---------------:|:--------:|:--------:|:-----------------:|:--------:|:---------------:|:------------:|:---------------:|:------------:|
-| 0.1822        | 1.0   | 30789 | 0.3109          | 0.8675   | 0.8798   | 0.8658            | 0.8798   | 0.8692          | 0.8658       | 0.8798          | 0.8798       |
-| 0.1334        | 2.0   | 61579 | 0.3410          | 0.8735   | 0.8860   | 0.8696            | 0.8860   | 0.8780          | 0.8696       | 0.8860          | 0.8860       |
-| 0.0829        | 3.0   | 92367 | 0.4505          | 0.8753   | 0.8869   | 0.8737            | 0.8869   | 0.8771          | 0.8737       | 0.8869          | 0.8869       |
-
-
-### Framework versions
-
-- Transformers 4.33.3
-- Pytorch 1.11.0+cu113
-- Datasets 2.14.6
-- Tokenizers 0.12.1
+The model is designed for zero-shot classification with the Hugging Face pipeline. 
+The model should be substantially better at zero-shot classification than my other zero-shot models on the
+Hugging Face hub: https://huggingface.co/MoritzLaurer.
+
+The model can do one universal task: determine whether a hypothesis is `true` or `not_true`
+given a text (also called `entailment` vs. `not_entailment`).  
+This task format is based on the Natural Language Inference task (NLI).
+The task is so universal that any classification task can be reformulated into the task.
+
+## Training data
+The model was trained on a mixture of 27 tasks and 310 classes that have been reformatted into this universal format.
+1.   26 classification tasks with ~400k texts:
+'amazonpolarity', 'imdb', 'appreviews', 'yelpreviews', 'rottentomatoes',
+'emotiondair', 'emocontext', 'empathetic',
+'financialphrasebank', 'banking77', 'massive',
+'wikitoxic_toxicaggregated', 'wikitoxic_obscene', 'wikitoxic_threat', 'wikitoxic_insult', 'wikitoxic_identityhate', 
+'hateoffensive', 'hatexplain', 'biasframes_offensive', 'biasframes_sex', 'biasframes_intent',
+'agnews', 'yahootopics',
+'trueteacher', 'spam', 'wellformedquery'.
+See details on each dataset here: https://docs.google.com/spreadsheets/d/1Z18tMh02IiWgh6o8pfoMiI_LH4IXpr78wd_nmNd5FaE/edit?usp=sharing
+3.   Five NLI datasets with ~885k texts: "mnli", "anli", "fever", "wanli", "ling"
+
+Note that compared to other NLI models, this model predicts two classes (`entailment` vs. `not_entailment`)
+as opposed to three classes (entailment/neutral/contradiction)
+
+
+### How to use the model
+#### Simple zero-shot classification pipeline
+```python
+from transformers import pipeline
+classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-large-zeroshot-v1")
+sequence_to_classify = "Angela Merkel is a politician in Germany and leader of the CDU"
+candidate_labels = ["politics", "economy", "entertainment", "environment"]
+output = classifier(sequence_to_classify, candidate_labels, multi_label=False)
+print(output)
+```
+
+### Details on data and training
+The code for preparing the data and training & evaluating the model is fully open-source here: https://github.com/MoritzLaurer/zeroshot-classifier/tree/main
+
+## Limitations and bias
+The model can only do text classification tasks. 
+
+Please consult the original DeBERTa paper and the papers for the different datasets for potential biases. 
+
+## License
+The base model (DeBERTa-v3) is published under the MIT license.
+The datasets the model was fine-tuned on are published under a diverse set of licenses.
+The following spreadsheet provides an overview of the non-NLI datasets used for fine-tuning.
+The spreadsheets contains information on licenses, the underlying papers etc.: https://docs.google.com/spreadsheets/d/1Z18tMh02IiWgh6o8pfoMiI_LH4IXpr78wd_nmNd5FaE/edit?usp=sharing
+
+In addition, the model was also trained on the following NLI datasets: MNLI, ANLI, WANLI, LING-NLI, FEVER-NLI.
+
+## Citation
+If you use this model, please cite: 
+```
+@article{laurer_less_2023,
+	title = {Less {Annotating}, {More} {Classifying}: {Addressing} the {Data} {Scarcity} {Issue} of {Supervised} {Machine} {Learning} with {Deep} {Transfer} {Learning} and {BERT}-{NLI}},
+	issn = {1047-1987, 1476-4989},
+	shorttitle = {Less {Annotating}, {More} {Classifying}},
+	url = {https://www.cambridge.org/core/product/identifier/S1047198723000207/type/journal_article},
+	doi = {10.1017/pan.2023.20},
+	language = {en},
+	urldate = {2023-06-20},
+	journal = {Political Analysis},
+	author = {Laurer, Moritz and Van Atteveldt, Wouter and Casas, Andreu and Welbers, Kasper},
+	month = jun,
+	year = {2023},
+	pages = {1--33},
+}
+```
+
+### Ideas for cooperation or questions?
+If you have questions or ideas for cooperation, contact me at m{dot}laurer{at}vu{dot}nl or [LinkedIn](https://www.linkedin.com/in/moritz-laurer/)
+
+### Debugging and issues
+Note that DeBERTa-v3 was released on 06.12.21 and older versions of HF Transformers seem to have issues running the model (e.g. resulting in an issue with the tokenizer). Using Transformers>=4.13 might solve some issues.