Upload with huggingface_hub

1_Pooling/config.json

@@ -0,0 +1,7 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": false,
+  "pooling_mode_mean_tokens": true,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false
+}

README.md

@@ -0,0 +1,129 @@
+---
+pipeline_tag: sentence-similarity
+tags:
+- sentence-transformers
+- feature-extraction
+- sentence-similarity
+- transformers
+
+---
+
+# {MODEL_NAME}
+
+This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
+
+<!--- Describe your model here -->
+
+## Usage (Sentence-Transformers)
+
+Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
+
+```
+pip install -U sentence-transformers
+```
+
+Then you can use the model like this:
+
+```python
+from sentence_transformers import SentenceTransformer
+sentences = ["This is an example sentence", "Each sentence is converted"]
+
+model = SentenceTransformer('{MODEL_NAME}')
+embeddings = model.encode(sentences)
+print(embeddings)
+```
+
+
+
+## Usage (HuggingFace Transformers)
+Without [sentence-transformers](https://www.SBERT.net), you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
+
+```python
+from transformers import AutoTokenizer, AutoModel
+import torch
+
+
+#Mean Pooling - Take attention mask into account for correct averaging
+def mean_pooling(model_output, attention_mask):
+    token_embeddings = model_output[0] #First element of model_output contains all token embeddings
+    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
+    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)
+
+
+# Sentences we want sentence embeddings for
+sentences = ['This is an example sentence', 'Each sentence is converted']
+
+# Load model from HuggingFace Hub
+tokenizer = AutoTokenizer.from_pretrained('{MODEL_NAME}')
+model = AutoModel.from_pretrained('{MODEL_NAME}')
+
+# Tokenize sentences
+encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
+
+# Compute token embeddings
+with torch.no_grad():
+    model_output = model(**encoded_input)
+
+# Perform pooling. In this case, mean pooling.
+sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])
+
+print("Sentence embeddings:")
+print(sentence_embeddings)
+```
+
+
+
+## Evaluation Results
+
+<!--- Describe how your model was evaluated -->
+
+For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})
+
+
+## Training
+The model was trained with the parameters:
+
+**DataLoader**:
+
+`torch.utils.data.dataloader.DataLoader` of length 2471 with parameters:
+```
+{'batch_size': 512, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
+```
+
+**Loss**:
+
+`sentence_transformers.losses.ContrastiveLoss.ContrastiveLoss` with parameters:
+  ```
+  {'distance_metric': 'SiameseDistanceMetric.COSINE_DISTANCE', 'margin': 0.5, 'size_average': True}
+  ```
+
+Parameters of the fit()-Method:
+```
+{
+    "epochs": 4,
+    "evaluation_steps": 1000,
+    "evaluator": "sentence_transformers.evaluation.BinaryClassificationEvaluator.BinaryClassificationEvaluator",
+    "max_grad_norm": 1,
+    "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
+    "optimizer_params": {
+        "lr": 2e-05
+    },
+    "scheduler": "WarmupLinear",
+    "steps_per_epoch": null,
+    "warmup_steps": 100,
+    "weight_decay": 0.01
+}
+```
+
+
+## Full Model Architecture
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
+)
+```
+
+## Citing & Authors
+
+<!--- Describe where people can find more information -->

config.json

@@ -0,0 +1,26 @@
+{
+  "_name_or_path": "/root/.cache/torch/sentence_transformers/bert-base-uncased",
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.20.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,7 @@
+{
+  "__version__": {
+    "sentence_transformers": "2.2.2",
+    "transformers": "4.20.1",
+    "pytorch": "1.11.0"
+  }
+}

eval/binary_classification_evaluation_results.csv

@@ -0,0 +1,13 @@
+epoch,steps,cossim_accuracy,cossim_accuracy_threshold,cossim_f1,cossim_precision,cossim_recall,cossim_f1_threshold,cossim_ap,manhattan_accuracy,manhattan_accuracy_threshold,manhattan_f1,manhattan_precision,manhattan_recall,manhattan_f1_threshold,manhattan_ap,euclidean_accuracy,euclidean_accuracy_threshold,euclidean_f1,euclidean_precision,euclidean_recall,euclidean_f1_threshold,euclidean_ap,dot_accuracy,dot_accuracy_threshold,dot_f1,dot_precision,dot_recall,dot_f1_threshold,dot_ap
+0,1000,0.9991462234617793,0.7837290167808533,0.42214532871972316,0.5041322314049587,0.3630952380952381,0.6963268518447876,0.33632485967798925,0.9991462234617793,143.71078491210938,0.3984375,0.5795454545454546,0.30357142857142855,169.0437469482422,0.3211073842702595,0.9991525477324328,6.5679731369018555,0.4046692607003891,0.5842696629213483,0.30952380952380953,7.691385269165039,0.32232224095594614,0.9990766564845909,95.89048767089844,0.34710743801652894,0.5675675675675675,0.25,86.46884155273438,0.2530268608263961
+0,2000,0.9991841690857002,0.8072477579116821,0.4569288389513108,0.6161616161616161,0.3630952380952381,0.7091373801231384,0.3835202624214063,0.9991715205443932,146.40200805664062,0.4290657439446367,0.512396694214876,0.36904761904761907,190.84320068359375,0.3579077782768174,0.9991778448150467,6.8259735107421875,0.4300341296928328,0.504,0.375,8.677935600280762,0.35820252595049207,0.9991462234617793,101.90943908691406,0.3891402714932127,0.8113207547169812,0.25595238095238093,101.29718780517578,0.3244226242237217
+0,-1,0.9992031418976607,0.7841047048568726,0.46875000000000006,0.6818181818181818,0.35714285714285715,0.7427619695663452,0.41115714941355436,0.9991904933563537,138.44175720214844,0.4573643410852713,0.6555555555555556,0.35119047619047616,160.37753295898438,0.3846922726834804,0.9991841690857002,6.348769187927246,0.45849802371541504,0.6823529411764706,0.34523809523809523,7.24355411529541,0.38465323982851785,0.9991272506498188,98.2314453125,0.3888888888888889,0.4666666666666667,0.3333333333333333,78.54522705078125,0.3307059303560602
+1,1000,0.9992474117922351,0.7920504808425903,0.4923076923076923,0.6956521739130435,0.38095238095238093,0.7131727933883667,0.4518931971830252,0.9992094661683142,159.49363708496094,0.5,0.67,0.39880952380952384,186.6231689453125,0.4285097852395977,0.9992157904389676,6.824808597564697,0.5,0.7272727272727273,0.38095238095238093,8.191187858581543,0.42940091423169935,0.9991651962737398,118.62881469726562,0.43018867924528303,0.5876288659793815,0.3392857142857143,92.50477600097656,0.3685279280743357
+1,2000,0.9992347632509281,0.7940564751625061,0.509933774834437,0.5746268656716418,0.4583333333333333,0.6585561037063599,0.4889158117267331,0.9992157904389676,173.400146484375,0.48648648648648646,0.6923076923076923,0.375,185.8468475341797,0.4354772896537532,0.9992157904389676,7.9467878341674805,0.48648648648648646,0.6923076923076923,0.375,8.428628921508789,0.4353464963653727,0.9992031418976607,105.52435302734375,0.48387096774193544,0.75,0.35714285714285715,96.61498260498047,0.41866355659256693
+1,-1,0.9992410875215816,0.7740247845649719,0.532871972318339,0.6363636363636364,0.4583333333333333,0.6593878269195557,0.5024074117701346,0.9992094661683142,175.75018310546875,0.5107913669064748,0.6454545454545455,0.4226190476190476,196.8307342529297,0.430273883513313,0.9992221147096211,8.076899528503418,0.5092250922509225,0.6699029126213593,0.4107142857142857,8.822940826416016,0.4354666541146464,0.9991904933563537,120.94625854492188,0.4573643410852713,0.6555555555555556,0.35119047619047616,103.39112091064453,0.3955497847369943
+2,1000,0.9992410875215816,0.8009589314460754,0.5256410256410257,0.5694444444444444,0.4880952380952381,0.6822413802146912,0.4993940447455595,0.9992410875215816,149.56777954101562,0.5147058823529412,0.6730769230769231,0.4166666666666667,192.71450805664062,0.4542026099409924,0.9992474117922351,6.889247417449951,0.5144927536231884,0.6574074074074074,0.4226190476190476,8.88015365600586,0.45771409905809207,0.9991904933563537,120.86624145507812,0.49134948096885817,0.5867768595041323,0.4226190476190476,103.25668334960938,0.42896053988137905
+2,2000,0.9993106544987699,0.7457864880561829,0.5657370517928286,0.8554216867469879,0.4226190476190476,0.7457864880561829,0.540502178151615,0.9992600603335421,173.0535888671875,0.5481481481481482,0.7254901960784313,0.44047619047619047,205.51473999023438,0.485027615674364,0.9992600603335421,7.862200736999512,0.5481481481481482,0.7254901960784313,0.44047619047619047,9.342029571533203,0.48690839132214747,0.9992031418976607,134.4933319091797,0.4980544747081712,0.7191011235955056,0.38095238095238093,120.86195373535156,0.4385379892277095
+2,-1,0.999285357416156,0.7436920404434204,0.5682656826568265,0.7475728155339806,0.4583333333333333,0.7037642002105713,0.5368994174943241,0.9992663846041955,183.12399291992188,0.5494505494505495,0.7142857142857143,0.44642857142857145,200.01278686523438,0.47195543059361267,0.9992663846041955,8.305182456970215,0.5494505494505495,0.7142857142857143,0.44642857142857145,9.049421310424805,0.4739523190528323,0.9992284389802746,128.38194274902344,0.5056603773584906,0.6907216494845361,0.39880952380952384,113.7187271118164,0.45520269474073893
+3,1000,0.9993169787694234,0.7334526777267456,0.5912408759124088,0.7641509433962265,0.48214285714285715,0.6974474191665649,0.5605111799599404,0.999272708874849,168.96597290039062,0.5571428571428572,0.6964285714285714,0.4642857142857143,219.14434814453125,0.5057631522525211,0.9992916816868095,9.036856651306152,0.5658914728682171,0.8111111111111111,0.43452380952380953,9.036856651306152,0.5078164920847897,0.9992916816868095,131.04042053222656,0.5503875968992248,0.7888888888888889,0.4226190476190476,124.12129211425781,0.4916428743723209
+3,2000,0.9993043302281165,0.7527890205383301,0.5931034482758619,0.7049180327868853,0.5119047619047619,0.6647562384605408,0.559255140250874,0.999285357416156,200.63621520996094,0.5615384615384615,0.7934782608695652,0.43452380952380953,202.9192352294922,0.5063478867694474,0.999285357416156,9.079340934753418,0.5615384615384615,0.7934782608695652,0.43452380952380953,9.195023536682129,0.5084140814403258,0.9992790331455025,132.403564453125,0.5475285171102662,0.7578947368421053,0.42857142857142855,125.06591796875,0.4946324778556296
+3,-1,0.9993043302281165,0.7540255188941956,0.5907473309608541,0.7345132743362832,0.49404761904761907,0.6928319931030273,0.5600199396114338,0.9992790331455025,194.93624877929688,0.5581395348837209,0.8,0.42857142857142855,199.81625366210938,0.5061753284729957,0.9992790331455025,8.704366683959961,0.5675675675675675,0.65625,0.5,10.60268783569336,0.5097813851011143,0.999285357416156,137.4734649658203,0.5461254612546126,0.7184466019417476,0.44047619047619047,123.53608703613281,0.4978411067534503

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:144f8c0537318547f248bc0665eb51779134a23c0770a76c157fcf263b43690d
+size 437998385

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "name_or_path": "/root/.cache/torch/sentence_transformers/bert-base-uncased",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "special_tokens_map_file": null,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}