Hugging Face's logo

kperkins411
/
msmarco-distilbert-base-v2_triplet_legal

Sentence Similarity
sentence-transformers
Safetensors
distilbert
feature-extraction
Generated from Trainer
dataset_size:88018
loss:TripletLoss
Eval Results
text-embeddings-inference
Inference Endpoints
Model card Files Community
kperkins411 commited on
Commit
791fc47
•
1 Parent(s): 1567d8d

Add new SentenceTransformer model.

Browse files
Files changed (2) hide show
  1. README.md +114 -88
  2. model.safetensors +1 -1
README.md CHANGED
@@ -1,4 +1,5 @@
1
  ---
 
2
  datasets: []
3
  language: []
4
  library_name: sentence-transformers
@@ -187,7 +188,7 @@ widget:
187
  and/or any of its affiliates and the directors, officers and employees of Domini
188
  and/or any of its affiliates.
189
  model-index:
190
- - name: SentenceTransformer
191
  results:
192
  - task:
193
  type: information-retrieval
@@ -200,103 +201,103 @@ model-index:
200
  value: 0.3953048087845513
201
  name: Cosine Accuracy@1
202
  - type: cosine_accuracy@3
203
- value: 0.5342673229837183
204
  name: Cosine Accuracy@3
205
  - type: cosine_accuracy@5
206
- value: 0.5914426353653919
207
  name: Cosine Accuracy@5
208
  - type: cosine_accuracy@10
209
- value: 0.66565694812571
210
  name: Cosine Accuracy@10
211
  - type: cosine_precision@1
212
  value: 0.3953048087845513
213
  name: Cosine Precision@1
214
  - type: cosine_precision@3
215
- value: 0.17808910766123942
216
  name: Cosine Precision@3
217
  - type: cosine_precision@5
218
- value: 0.11828852707307837
219
  name: Cosine Precision@5
220
  - type: cosine_precision@10
221
- value: 0.06656569481257099
222
  name: Cosine Precision@10
223
  - type: cosine_recall@1
224
  value: 0.3953048087845513
225
  name: Cosine Recall@1
226
  - type: cosine_recall@3
227
- value: 0.5342673229837183
228
  name: Cosine Recall@3
229
  - type: cosine_recall@5
230
- value: 0.5914426353653919
231
  name: Cosine Recall@5
232
  - type: cosine_recall@10
233
- value: 0.66565694812571
234
  name: Cosine Recall@10
235
  - type: cosine_ndcg@10
236
- value: 0.5240873176000084
237
  name: Cosine Ndcg@10
238
  - type: cosine_mrr@10
239
- value: 0.4794995582481382
240
  name: Cosine Mrr@10
241
  - type: cosine_map@100
242
- value: 0.4872380542829767
243
  name: Cosine Map@100
244
  - type: dot_accuracy@1
245
- value: 0.3934115865202575
246
  name: Dot Accuracy@1
247
  - type: dot_accuracy@3
248
- value: 0.5312381673608482
249
  name: Dot Accuracy@3
250
  - type: dot_accuracy@5
251
- value: 0.5899280575539568
252
  name: Dot Accuracy@5
253
  - type: dot_accuracy@10
254
- value: 0.6648996592199924
255
  name: Dot Accuracy@10
256
  - type: dot_precision@1
257
- value: 0.3934115865202575
258
  name: Dot Precision@1
259
  - type: dot_precision@3
260
- value: 0.1770793891202827
261
  name: Dot Precision@3
262
  - type: dot_precision@5
263
- value: 0.11798561151079137
264
  name: Dot Precision@5
265
  - type: dot_precision@10
266
- value: 0.06648996592199924
267
  name: Dot Precision@10
268
  - type: dot_recall@1
269
- value: 0.3934115865202575
270
  name: Dot Recall@1
271
  - type: dot_recall@3
272
- value: 0.5312381673608482
273
  name: Dot Recall@3
274
  - type: dot_recall@5
275
- value: 0.5899280575539568
276
  name: Dot Recall@5
277
  - type: dot_recall@10
278
- value: 0.6648996592199924
279
  name: Dot Recall@10
280
  - type: dot_ndcg@10
281
- value: 0.5224316548033627
282
  name: Dot Ndcg@10
283
  - type: dot_mrr@10
284
- value: 0.4775905591316421
285
  name: Dot Mrr@10
286
  - type: dot_map@100
287
- value: 0.485319730256097
288
  name: Dot Map@100
289
  ---
290
 
291
- # SentenceTransformer
292
 
293
- This is a [sentence-transformers](https://www.SBERT.net) model trained. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
294
 
295
  ## Model Details
296
 
297
  ### Model Description
298
  - **Model Type:** Sentence Transformer
299
- <!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
300
  - **Maximum Sequence Length:** 350 tokens
301
  - **Output Dimensionality:** 768 tokens
302
  - **Similarity Function:** Cosine Similarity
@@ -383,38 +384,38 @@ You can finetune this model on your own dataset.
383
  * Dataset: `msmarco-distilbert-base-v2`
384
  * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
385
 
386
- | Metric | Value |
387
- |:--------------------|:-----------|
388
- | cosine_accuracy@1 | 0.3953 |
389
- | cosine_accuracy@3 | 0.5343 |
390
- | cosine_accuracy@5 | 0.5914 |
391
- | cosine_accuracy@10 | 0.6657 |
392
- | cosine_precision@1 | 0.3953 |
393
- | cosine_precision@3 | 0.1781 |
394
- | cosine_precision@5 | 0.1183 |
395
- | cosine_precision@10 | 0.0666 |
396
- | cosine_recall@1 | 0.3953 |
397
- | cosine_recall@3 | 0.5343 |
398
- | cosine_recall@5 | 0.5914 |
399
- | cosine_recall@10 | 0.6657 |
400
- | cosine_ndcg@10 | 0.5241 |
401
- | cosine_mrr@10 | 0.4795 |
402
- | **cosine_map@100** | **0.4872** |
403
- | dot_accuracy@1 | 0.3934 |
404
- | dot_accuracy@3 | 0.5312 |
405
- | dot_accuracy@5 | 0.5899 |
406
- | dot_accuracy@10 | 0.6649 |
407
- | dot_precision@1 | 0.3934 |
408
- | dot_precision@3 | 0.1771 |
409
- | dot_precision@5 | 0.118 |
410
- | dot_precision@10 | 0.0665 |
411
- | dot_recall@1 | 0.3934 |
412
- | dot_recall@3 | 0.5312 |
413
- | dot_recall@5 | 0.5899 |
414
- | dot_recall@10 | 0.6649 |
415
- | dot_ndcg@10 | 0.5224 |
416
- | dot_mrr@10 | 0.4776 |
417
- | dot_map@100 | 0.4853 |
418
 
419
  <!--
420
  ## Bias, Risks and Limitations
@@ -489,6 +490,7 @@ You can finetune this model on your own dataset.
489
  - `per_device_train_batch_size`: 128
490
  - `per_device_eval_batch_size`: 128
491
  - `learning_rate`: 2e-05
 
492
  - `warmup_ratio`: 0.1
493
  - `fp16`: True
494
  - `load_best_model_at_end`: True
@@ -513,7 +515,7 @@ You can finetune this model on your own dataset.
513
  - `adam_beta2`: 0.999
514
  - `adam_epsilon`: 1e-08
515
  - `max_grad_norm`: 1.0
516
- - `num_train_epochs`: 3
517
  - `max_steps`: -1
518
  - `lr_scheduler_type`: linear
519
  - `lr_scheduler_kwargs`: {}
@@ -611,30 +613,54 @@ You can finetune this model on your own dataset.
611
  ### Training Logs
612
  | Epoch | Step | Training Loss | loss | msmarco-distilbert-base-v2_cosine_map@100 |
613
  |:----------:|:--------:|:-------------:|:----------:|:-----------------------------------------:|
614
- | 0 | 0 | - | - | 0.4899 |
615
- | 0.1453 | 100 | 0.0787 | - | - |
616
- | 0.2907 | 200 | 0.0503 | - | - |
617
- | 0.4360 | 300 | 0.0529 | - | - |
618
- | 0.5814 | 400 | 0.0636 | - | - |
619
- | 0.7267 | 500 | 0.0783 | - | - |
620
- | 0.8721 | 600 | 0.0765 | - | - |
621
- | 1.0131 | 697 | - | 0.2284 | - |
622
- | 1.0044 | 700 | 0.0776 | - | - |
623
- | 1.1497 | 800 | 0.0624 | - | - |
624
- | 1.2951 | 900 | 0.0289 | - | - |
625
- | 1.4404 | 1000 | 0.0244 | - | - |
626
- | 1.5858 | 1100 | 0.0256 | - | - |
627
- | 1.7311 | 1200 | 0.0364 | - | - |
628
- | 1.8765 | 1300 | 0.0334 | - | - |
629
- | 2.0131 | 1394 | - | 0.2175 | - |
630
- | 2.0087 | 1400 | 0.0342 | - | - |
631
- | 2.1541 | 1500 | 0.0274 | - | - |
632
- | 2.2994 | 1600 | 0.0153 | - | - |
633
- | 2.4448 | 1700 | 0.0167 | - | - |
634
- | 2.5901 | 1800 | 0.0178 | - | - |
635
- | 2.7355 | 1900 | 0.0221 | - | - |
636
- | 2.8808 | 2000 | 0.0227 | - | - |
637
- | **2.9738** | **2064** | **-** | **0.1821** | **0.4872** |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
638
 
639
  * The bold row denotes the saved checkpoint.
640
 
 
1
  ---
2
+ base_model: sentence-transformers/msmarco-distilbert-base-v2
3
  datasets: []
4
  language: []
5
  library_name: sentence-transformers
 
188
  and/or any of its affiliates and the directors, officers and employees of Domini
189
  and/or any of its affiliates.
190
  model-index:
191
+ - name: SentenceTransformer based on sentence-transformers/msmarco-distilbert-base-v2
192
  results:
193
  - task:
194
  type: information-retrieval
 
201
  value: 0.3953048087845513
202
  name: Cosine Accuracy@1
203
  - type: cosine_accuracy@3
204
+ value: 0.5376751230594472
205
  name: Cosine Accuracy@3
206
  - type: cosine_accuracy@5
207
+ value: 0.594471790988262
208
  name: Cosine Accuracy@5
209
  - type: cosine_accuracy@10
210
+ value: 0.673608481635744
211
  name: Cosine Accuracy@10
212
  - type: cosine_precision@1
213
  value: 0.3953048087845513
214
  name: Cosine Precision@1
215
  - type: cosine_precision@3
216
+ value: 0.1792250410198157
217
  name: Cosine Precision@3
218
  - type: cosine_precision@5
219
+ value: 0.1188943581976524
220
  name: Cosine Precision@5
221
  - type: cosine_precision@10
222
+ value: 0.06736084816357439
223
  name: Cosine Precision@10
224
  - type: cosine_recall@1
225
  value: 0.3953048087845513
226
  name: Cosine Recall@1
227
  - type: cosine_recall@3
228
+ value: 0.5376751230594472
229
  name: Cosine Recall@3
230
  - type: cosine_recall@5
231
+ value: 0.594471790988262
232
  name: Cosine Recall@5
233
  - type: cosine_recall@10
234
+ value: 0.673608481635744
235
  name: Cosine Recall@10
236
  - type: cosine_ndcg@10
237
+ value: 0.5276829229789854
238
  name: Cosine Ndcg@10
239
  - type: cosine_mrr@10
240
+ value: 0.4818510605049796
241
  name: Cosine Mrr@10
242
  - type: cosine_map@100
243
+ value: 0.48897515764559735
244
  name: Cosine Map@100
245
  - type: dot_accuracy@1
246
+ value: 0.3964407421431276
247
  name: Dot Accuracy@1
248
  - type: dot_accuracy@3
249
+ value: 0.5335100340780008
250
  name: Dot Accuracy@3
251
  - type: dot_accuracy@5
252
+ value: 0.5933358576296858
253
  name: Dot Accuracy@5
254
  - type: dot_accuracy@10
255
+ value: 0.6743657705414615
256
  name: Dot Accuracy@10
257
  - type: dot_precision@1
258
+ value: 0.3964407421431276
259
  name: Dot Precision@1
260
  - type: dot_precision@3
261
+ value: 0.17783667802600023
262
  name: Dot Precision@3
263
  - type: dot_precision@5
264
+ value: 0.11866717152593716
265
  name: Dot Precision@5
266
  - type: dot_precision@10
267
+ value: 0.06743657705414616
268
  name: Dot Precision@10
269
  - type: dot_recall@1
270
+ value: 0.3964407421431276
271
  name: Dot Recall@1
272
  - type: dot_recall@3
273
+ value: 0.5335100340780008
274
  name: Dot Recall@3
275
  - type: dot_recall@5
276
+ value: 0.5933358576296858
277
  name: Dot Recall@5
278
  - type: dot_recall@10
279
+ value: 0.6743657705414615
280
  name: Dot Recall@10
281
  - type: dot_ndcg@10
282
+ value: 0.5274757216450244
283
  name: Dot Ndcg@10
284
  - type: dot_mrr@10
285
+ value: 0.4814724160521211
286
  name: Dot Mrr@10
287
  - type: dot_map@100
288
+ value: 0.4884569183065979
289
  name: Dot Map@100
290
  ---
291
 
292
+ # SentenceTransformer based on sentence-transformers/msmarco-distilbert-base-v2
293
 
294
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/msmarco-distilbert-base-v2](https://huggingface.co/sentence-transformers/msmarco-distilbert-base-v2). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
295
 
296
  ## Model Details
297
 
298
  ### Model Description
299
  - **Model Type:** Sentence Transformer
300
+ - **Base model:** [sentence-transformers/msmarco-distilbert-base-v2](https://huggingface.co/sentence-transformers/msmarco-distilbert-base-v2) <!-- at revision 741fcf2d6eabaf0927bfe49c6d9c577df95d3c40 -->
301
  - **Maximum Sequence Length:** 350 tokens
302
  - **Output Dimensionality:** 768 tokens
303
  - **Similarity Function:** Cosine Similarity
 
384
  * Dataset: `msmarco-distilbert-base-v2`
385
  * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
386
 
387
+ | Metric | Value |
388
+ |:--------------------|:----------|
389
+ | cosine_accuracy@1 | 0.3953 |
390
+ | cosine_accuracy@3 | 0.5377 |
391
+ | cosine_accuracy@5 | 0.5945 |
392
+ | cosine_accuracy@10 | 0.6736 |
393
+ | cosine_precision@1 | 0.3953 |
394
+ | cosine_precision@3 | 0.1792 |
395
+ | cosine_precision@5 | 0.1189 |
396
+ | cosine_precision@10 | 0.0674 |
397
+ | cosine_recall@1 | 0.3953 |
398
+ | cosine_recall@3 | 0.5377 |
399
+ | cosine_recall@5 | 0.5945 |
400
+ | cosine_recall@10 | 0.6736 |
401
+ | cosine_ndcg@10 | 0.5277 |
402
+ | cosine_mrr@10 | 0.4819 |
403
+ | **cosine_map@100** | **0.489** |
404
+ | dot_accuracy@1 | 0.3964 |
405
+ | dot_accuracy@3 | 0.5335 |
406
+ | dot_accuracy@5 | 0.5933 |
407
+ | dot_accuracy@10 | 0.6744 |
408
+ | dot_precision@1 | 0.3964 |
409
+ | dot_precision@3 | 0.1778 |
410
+ | dot_precision@5 | 0.1187 |
411
+ | dot_precision@10 | 0.0674 |
412
+ | dot_recall@1 | 0.3964 |
413
+ | dot_recall@3 | 0.5335 |
414
+ | dot_recall@5 | 0.5933 |
415
+ | dot_recall@10 | 0.6744 |
416
+ | dot_ndcg@10 | 0.5275 |
417
+ | dot_mrr@10 | 0.4815 |
418
+ | dot_map@100 | 0.4885 |
419
 
420
  <!--
421
  ## Bias, Risks and Limitations
 
490
  - `per_device_train_batch_size`: 128
491
  - `per_device_eval_batch_size`: 128
492
  - `learning_rate`: 2e-05
493
+ - `num_train_epochs`: 6
494
  - `warmup_ratio`: 0.1
495
  - `fp16`: True
496
  - `load_best_model_at_end`: True
 
515
  - `adam_beta2`: 0.999
516
  - `adam_epsilon`: 1e-08
517
  - `max_grad_norm`: 1.0
518
+ - `num_train_epochs`: 6
519
  - `max_steps`: -1
520
  - `lr_scheduler_type`: linear
521
  - `lr_scheduler_kwargs`: {}
 
613
  ### Training Logs
614
  | Epoch | Step | Training Loss | loss | msmarco-distilbert-base-v2_cosine_map@100 |
615
  |:----------:|:--------:|:-------------:|:----------:|:-----------------------------------------:|
616
+ | 0 | 0 | - | - | 0.4145 |
617
+ | 0.1453 | 100 | 1.7626 | - | - |
618
+ | 0.2907 | 200 | 0.9595 | - | - |
619
+ | 0.4360 | 300 | 0.7263 | - | - |
620
+ | 0.5814 | 400 | 0.6187 | - | - |
621
+ | 0.7267 | 500 | 0.5571 | - | - |
622
+ | 0.8721 | 600 | 0.4885 | - | - |
623
+ | 1.0131 | 697 | - | 0.3676 | - |
624
+ | 1.0044 | 700 | 0.4283 | - | - |
625
+ | 1.1497 | 800 | 0.3956 | - | - |
626
+ | 1.2951 | 900 | 0.2941 | - | - |
627
+ | 1.4404 | 1000 | 0.2437 | - | - |
628
+ | 1.5858 | 1100 | 0.1988 | - | - |
629
+ | 1.7311 | 1200 | 0.185 | - | - |
630
+ | 1.8765 | 1300 | 0.1571 | - | - |
631
+ | 2.0131 | 1394 | - | 0.2679 | - |
632
+ | 2.0087 | 1400 | 0.1409 | - | - |
633
+ | 2.1541 | 1500 | 0.1368 | - | - |
634
+ | 2.2994 | 1600 | 0.111 | - | - |
635
+ | 2.4448 | 1700 | 0.0994 | - | - |
636
+ | 2.5901 | 1800 | 0.0837 | - | - |
637
+ | 2.7355 | 1900 | 0.076 | - | - |
638
+ | 2.8808 | 2000 | 0.0645 | - | - |
639
+ | 3.0131 | 2091 | - | 0.2412 | - |
640
+ | 3.0131 | 2100 | 0.0607 | - | - |
641
+ | 3.1584 | 2200 | 0.0609 | - | - |
642
+ | 3.3038 | 2300 | 0.0503 | - | - |
643
+ | 3.4491 | 2400 | 0.0483 | - | - |
644
+ | 3.5945 | 2500 | 0.0402 | - | - |
645
+ | 3.7398 | 2600 | 0.0397 | - | - |
646
+ | 3.8852 | 2700 | 0.0305 | - | - |
647
+ | 4.0131 | 2788 | - | 0.2196 | - |
648
+ | 4.0174 | 2800 | 0.0304 | - | - |
649
+ | 4.1628 | 2900 | 0.0307 | - | - |
650
+ | 4.3081 | 3000 | 0.0256 | - | - |
651
+ | 4.4535 | 3100 | 0.0258 | - | - |
652
+ | 4.5988 | 3200 | 0.0212 | - | - |
653
+ | 4.7442 | 3300 | 0.0213 | - | - |
654
+ | 4.8895 | 3400 | 0.0174 | - | - |
655
+ | 5.0131 | 3485 | - | 0.2036 | - |
656
+ | 5.0218 | 3500 | 0.0191 | - | - |
657
+ | 5.1672 | 3600 | 0.0198 | - | - |
658
+ | 5.3125 | 3700 | 0.0161 | - | - |
659
+ | 5.4578 | 3800 | 0.0166 | - | - |
660
+ | 5.6032 | 3900 | 0.0135 | - | - |
661
+ | 5.7485 | 4000 | 0.0145 | - | - |
662
+ | 5.8939 | 4100 | 0.0129 | - | - |
663
+ | **5.9346** | **4128** | **-** | **0.1966** | **0.489** |
664
 
665
  * The bold row denotes the saved checkpoint.
666
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1796c6e005742413b753de6f83fdd6c3515b94cb1fce753d6adae3c90fe9191d
3
  size 265462608
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ff4f47578afdd7445b15b66710dfe43895a5be76181400182d87f9d1700cd4f
3
  size 265462608