Add SetFit model

README.md

@@ -1,126 +1,49 @@
 ---
-pipeline_tag: sentence-similarity
+license: apache-2.0
 tags:
+- setfit
 - sentence-transformers
-- feature-extraction
-- sentence-similarity
-- transformers
-
+- text-classification
+pipeline_tag: text-classification
 ---
 
-# {MODEL_NAME}
-
-This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
-
-<!--- Describe your model here -->
+# lewtun/my-awesome-setfit-model
 
-## Usage (Sentence-Transformers)
+This is a [SetFit model](https://github.com/huggingface/setfit) that can be used for text classification. The model has been trained using an efficient few-shot learning technique that involves:
 
-Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
-
-```
-pip install -U sentence-transformers
-```
+1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
+2. Training a classification head with features from the fine-tuned Sentence Transformer.
 
-Then you can use the model like this:
+## Usage
 
-```python
-from sentence_transformers import SentenceTransformer
-sentences = ["This is an example sentence", "Each sentence is converted"]
+To use this model for inference, first install the SetFit library:
 
-model = SentenceTransformer('{MODEL_NAME}')
-embeddings = model.encode(sentences)
-print(embeddings)
+```bash
+python -m pip install setfit
 ```
 
-
-
-## Usage (HuggingFace Transformers)
-Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
+You can then run inference as follows:
 
 ```python
-from transformers import AutoTokenizer, AutoModel
-import torch
-
-
-#Mean Pooling - Take attention mask into account for correct averaging
-def mean_pooling(model_output, attention_mask):
- token_embeddings = model_output[0] #First element of model_output contains all token embeddings
- input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
- return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)
-
-
-# Sentences we want sentence embeddings for
-sentences = ['This is an example sentence', 'Each sentence is converted']
-
-# Load model from HuggingFace Hub
-tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
-model = AutoModel.from_pretrained('{MODEL_NAME}')
-
-# Tokenize sentences
-encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
-
-# Compute token embeddings
-with torch.no_grad():
- model_output = model(**encoded_input)
-
-# Perform pooling. In this case, mean pooling.
-sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
-
-print("Sentence embeddings:")
-print(sentence_embeddings)
-```
-
-
-
-## Evaluation Results
-
-<!--- Describe how your model was evaluated -->
-
-For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})
-
-
-## Training
-The model was trained with the parameters:
-
-**DataLoader**:
-
-`torch.utils.data.dataloader.DataLoader` of length 40 with parameters:
-```
-{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
-```
-
-**Loss**:
-
-`sentence_transformers.losses.CosineSimilarityLoss.CosineSimilarityLoss` 
-
-Parameters of the fit()-Method:
-```
-{
- "epochs": 1,
- "evaluation_steps": 0,
- "evaluator": "NoneType",
- "max_grad_norm": 1,
- "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
- "optimizer_params": {
- "lr": 2e-05
- },
- "scheduler": "WarmupLinear",
- "steps_per_epoch": 40,
- "warmup_steps": 4,
- "weight_decay": 0.01
+from setfit import SetFitModel
+
+# Download from Hub and run inference
+model = SetFitModel.from_pretrained("lewtun/my-awesome-setfit-model")
+# Run inference
+preds = model(["i loved the spiderman movie!", "pineapple on pizza is the worst 🤮"])
+```
+
+## BibTeX entry and citation info
+
+```bibtex
+@article{https://doi.org/10.48550/arxiv.2209.11055,
+doi = {10.48550/ARXIV.2209.11055},
+url = {https://arxiv.org/abs/2209.11055},
+author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
+keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+title = {Efficient Few-Shot Learning Without Prompts},
+publisher = {arXiv},
+year = {2022},
+copyright = {Creative Commons Attribution 4.0 International}
 }
 ```
-
-
-## Full Model Architecture
-```
-SentenceTransformer(
- (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
- (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
-)
-```
-
-## Citing & Authors
-
-<!--- Describe where people can find more information -->

config.json

@@ -1,5 +1,5 @@
 {
- "_name_or_path": "/home/lewis_huggingface_co/.cache/torch/sentence_transformers/sentence-transformers_paraphrase-mpnet-base-v2/",
+ "_name_or_path": "/home/zjs/.cache/torch/sentence_transformers/sentence-transformers_paraphrase-mpnet-base-v2/",
  "architectures": [
  "MPNetModel"
  ],
@@ -19,6 +19,6 @@
  "pad_token_id": 1,
  "relative_attention_num_buckets": 32,
  "torch_dtype": "float32",
- "transformers_version": "4.20.0",
+ "transformers_version": "4.30.2",
  "vocab_size": 30527
 }

model_head.pkl

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6802cd87b7c943c0591009cb3bc4063470757fa53ed89b600f3f2a819e7b0d42
-size 6927
+oid sha256:38f4c170e3f7734344bce3c15bdbe2142484e9253b18fdf723b937aa234b8d8e
+size 7041

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:acb2bffff89914827e11045bce312c1f74eddaeb2bfdb4bfa9b78ab132cb854e
-size 438014769
+oid sha256:4be98415bdf9d33cd6fa30ffc65e831b33fff0d5196a5c386f4f3ed84713ed0c
+size 438016493

tokenizer_config.json

@@ -7,6 +7,7 @@
  "rstrip": false,
  "single_word": false
  },
+ "clean_up_tokenization_spaces": true,
  "cls_token": {
  "__type": "AddedToken",
  "content": "<s>",
@@ -34,7 +35,6 @@
  "single_word": false
  },
  "model_max_length": 512,
- "name_or_path": "/home/lewis_huggingface_co/.cache/torch/sentence_transformers/sentence-transformers_paraphrase-mpnet-base-v2/",
  "never_split": null,
  "pad_token": {
  "__type": "AddedToken",
@@ -52,7 +52,6 @@
  "rstrip": false,
  "single_word": false
  },
- "special_tokens_map_file": null,
  "strip_accents": null,
  "tokenize_chinese_chars": true,
  "tokenizer_class": "MPNetTokenizer",