rohitashva commited on
Commit
2e33b41
1 Parent(s): 14e6375

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +81 -2567
README.md CHANGED
@@ -12,2586 +12,100 @@ tags:
12
  - dementia
13
  - dementia disease
14
  language: en
15
- inference: false
16
  license: apache-2.0
17
- model-index:
18
- - name: INSTRUCTOR
19
- results:
20
- - task:
21
- type: Classification
22
- dataset:
23
- type: mteb/amazon_counterfactual
24
- name: MTEB AmazonCounterfactualClassification (en)
25
- config: en
26
- split: test
27
- revision: e8379541af4e31359cca9fbcf4b00f2671dba205
28
- metrics:
29
- - type: accuracy
30
- value: 88.13432835820896
31
- - type: ap
32
- value: 59.298209334395665
33
- - type: f1
34
- value: 83.31769058643586
35
- - task:
36
- type: Classification
37
- dataset:
38
- type: mteb/amazon_polarity
39
- name: MTEB AmazonPolarityClassification
40
- config: default
41
- split: test
42
- revision: e2d317d38cd51312af73b3d32a06d1a08b442046
43
- metrics:
44
- - type: accuracy
45
- value: 91.526375
46
- - type: ap
47
- value: 88.16327709705504
48
- - type: f1
49
- value: 91.51095801287843
50
- - task:
51
- type: Classification
52
- dataset:
53
- type: mteb/amazon_reviews_multi
54
- name: MTEB AmazonReviewsClassification (en)
55
- config: en
56
- split: test
57
- revision: 1399c76144fd37290681b995c656ef9b2e06e26d
58
- metrics:
59
- - type: accuracy
60
- value: 47.856
61
- - type: f1
62
- value: 45.41490917650942
63
- - task:
64
- type: Retrieval
65
- dataset:
66
- type: arguana
67
- name: MTEB ArguAna
68
- config: default
69
- split: test
70
- revision: None
71
- metrics:
72
- - type: map_at_1
73
- value: 31.223
74
- - type: map_at_10
75
- value: 47.947
76
- - type: map_at_100
77
- value: 48.742000000000004
78
- - type: map_at_1000
79
- value: 48.745
80
- - type: map_at_3
81
- value: 43.137
82
- - type: map_at_5
83
- value: 45.992
84
- - type: mrr_at_1
85
- value: 32.432
86
- - type: mrr_at_10
87
- value: 48.4
88
- - type: mrr_at_100
89
- value: 49.202
90
- - type: mrr_at_1000
91
- value: 49.205
92
- - type: mrr_at_3
93
- value: 43.551
94
- - type: mrr_at_5
95
- value: 46.467999999999996
96
- - type: ndcg_at_1
97
- value: 31.223
98
- - type: ndcg_at_10
99
- value: 57.045
100
- - type: ndcg_at_100
101
- value: 60.175
102
- - type: ndcg_at_1000
103
- value: 60.233000000000004
104
- - type: ndcg_at_3
105
- value: 47.171
106
- - type: ndcg_at_5
107
- value: 52.322
108
- - type: precision_at_1
109
- value: 31.223
110
- - type: precision_at_10
111
- value: 8.599
112
- - type: precision_at_100
113
- value: 0.991
114
- - type: precision_at_1000
115
- value: 0.1
116
- - type: precision_at_3
117
- value: 19.63
118
- - type: precision_at_5
119
- value: 14.282
120
- - type: recall_at_1
121
- value: 31.223
122
- - type: recall_at_10
123
- value: 85.989
124
- - type: recall_at_100
125
- value: 99.075
126
- - type: recall_at_1000
127
- value: 99.502
128
- - type: recall_at_3
129
- value: 58.89
130
- - type: recall_at_5
131
- value: 71.408
132
- - task:
133
- type: Clustering
134
- dataset:
135
- type: mteb/arxiv-clustering-p2p
136
- name: MTEB ArxivClusteringP2P
137
- config: default
138
- split: test
139
- revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
140
- metrics:
141
- - type: v_measure
142
- value: 43.1621946393635
143
- - task:
144
- type: Clustering
145
- dataset:
146
- type: mteb/arxiv-clustering-s2s
147
- name: MTEB ArxivClusteringS2S
148
- config: default
149
- split: test
150
- revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
151
- metrics:
152
- - type: v_measure
153
- value: 32.56417132407894
154
- - task:
155
- type: Reranking
156
- dataset:
157
- type: mteb/askubuntudupquestions-reranking
158
- name: MTEB AskUbuntuDupQuestions
159
- config: default
160
- split: test
161
- revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
162
- metrics:
163
- - type: map
164
- value: 64.29539304390207
165
- - type: mrr
166
- value: 76.44484017060196
167
- - task:
168
- type: STS
169
- dataset:
170
- type: mteb/biosses-sts
171
- name: MTEB BIOSSES
172
- config: default
173
- split: test
174
- revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
175
- metrics:
176
- - type: cos_sim_spearman
177
- value: 84.38746499431112
178
- - task:
179
- type: Classification
180
- dataset:
181
- type: mteb/banking77
182
- name: MTEB Banking77Classification
183
- config: default
184
- split: test
185
- revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
186
- metrics:
187
- - type: accuracy
188
- value: 78.51298701298701
189
- - type: f1
190
- value: 77.49041754069235
191
- - task:
192
- type: Clustering
193
- dataset:
194
- type: mteb/biorxiv-clustering-p2p
195
- name: MTEB BiorxivClusteringP2P
196
- config: default
197
- split: test
198
- revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
199
- metrics:
200
- - type: v_measure
201
- value: 37.61848554098577
202
- - task:
203
- type: Clustering
204
- dataset:
205
- type: mteb/biorxiv-clustering-s2s
206
- name: MTEB BiorxivClusteringS2S
207
- config: default
208
- split: test
209
- revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
210
- metrics:
211
- - type: v_measure
212
- value: 31.32623280148178
213
- - task:
214
- type: Retrieval
215
- dataset:
216
- type: BeIR/cqadupstack
217
- name: MTEB CQADupstackAndroidRetrieval
218
- config: default
219
- split: test
220
- revision: None
221
- metrics:
222
- - type: map_at_1
223
- value: 35.803000000000004
224
- - type: map_at_10
225
- value: 48.848
226
- - type: map_at_100
227
- value: 50.5
228
- - type: map_at_1000
229
- value: 50.602999999999994
230
- - type: map_at_3
231
- value: 45.111000000000004
232
- - type: map_at_5
233
- value: 47.202
234
- - type: mrr_at_1
235
- value: 44.635000000000005
236
- - type: mrr_at_10
237
- value: 55.593
238
- - type: mrr_at_100
239
- value: 56.169999999999995
240
- - type: mrr_at_1000
241
- value: 56.19499999999999
242
- - type: mrr_at_3
243
- value: 53.361999999999995
244
- - type: mrr_at_5
245
- value: 54.806999999999995
246
- - type: ndcg_at_1
247
- value: 44.635000000000005
248
- - type: ndcg_at_10
249
- value: 55.899
250
- - type: ndcg_at_100
251
- value: 60.958
252
- - type: ndcg_at_1000
253
- value: 62.302
254
- - type: ndcg_at_3
255
- value: 51.051
256
- - type: ndcg_at_5
257
- value: 53.351000000000006
258
- - type: precision_at_1
259
- value: 44.635000000000005
260
- - type: precision_at_10
261
- value: 10.786999999999999
262
- - type: precision_at_100
263
- value: 1.6580000000000001
264
- - type: precision_at_1000
265
- value: 0.213
266
- - type: precision_at_3
267
- value: 24.893
268
- - type: precision_at_5
269
- value: 17.740000000000002
270
- - type: recall_at_1
271
- value: 35.803000000000004
272
- - type: recall_at_10
273
- value: 68.657
274
- - type: recall_at_100
275
- value: 89.77199999999999
276
- - type: recall_at_1000
277
- value: 97.67
278
- - type: recall_at_3
279
- value: 54.066
280
- - type: recall_at_5
281
- value: 60.788
282
- - task:
283
- type: Retrieval
284
- dataset:
285
- type: BeIR/cqadupstack
286
- name: MTEB CQADupstackEnglishRetrieval
287
- config: default
288
- split: test
289
- revision: None
290
- metrics:
291
- - type: map_at_1
292
- value: 33.706
293
- - type: map_at_10
294
- value: 44.896
295
- - type: map_at_100
296
- value: 46.299
297
- - type: map_at_1000
298
- value: 46.44
299
- - type: map_at_3
300
- value: 41.721000000000004
301
- - type: map_at_5
302
- value: 43.486000000000004
303
- - type: mrr_at_1
304
- value: 41.592
305
- - type: mrr_at_10
306
- value: 50.529
307
- - type: mrr_at_100
308
- value: 51.22
309
- - type: mrr_at_1000
310
- value: 51.258
311
- - type: mrr_at_3
312
- value: 48.205999999999996
313
- - type: mrr_at_5
314
- value: 49.528
315
- - type: ndcg_at_1
316
- value: 41.592
317
- - type: ndcg_at_10
318
- value: 50.77199999999999
319
- - type: ndcg_at_100
320
- value: 55.383
321
- - type: ndcg_at_1000
322
- value: 57.288
323
- - type: ndcg_at_3
324
- value: 46.324
325
- - type: ndcg_at_5
326
- value: 48.346000000000004
327
- - type: precision_at_1
328
- value: 41.592
329
- - type: precision_at_10
330
- value: 9.516
331
- - type: precision_at_100
332
- value: 1.541
333
- - type: precision_at_1000
334
- value: 0.2
335
- - type: precision_at_3
336
- value: 22.399
337
- - type: precision_at_5
338
- value: 15.770999999999999
339
- - type: recall_at_1
340
- value: 33.706
341
- - type: recall_at_10
342
- value: 61.353
343
- - type: recall_at_100
344
- value: 80.182
345
- - type: recall_at_1000
346
- value: 91.896
347
- - type: recall_at_3
348
- value: 48.204
349
- - type: recall_at_5
350
- value: 53.89699999999999
351
- - task:
352
- type: Retrieval
353
- dataset:
354
- type: BeIR/cqadupstack
355
- name: MTEB CQADupstackGamingRetrieval
356
- config: default
357
- split: test
358
- revision: None
359
- metrics:
360
- - type: map_at_1
361
- value: 44.424
362
- - type: map_at_10
363
- value: 57.169000000000004
364
- - type: map_at_100
365
- value: 58.202
366
- - type: map_at_1000
367
- value: 58.242000000000004
368
- - type: map_at_3
369
- value: 53.825
370
- - type: map_at_5
371
- value: 55.714
372
- - type: mrr_at_1
373
- value: 50.470000000000006
374
- - type: mrr_at_10
375
- value: 60.489000000000004
376
- - type: mrr_at_100
377
- value: 61.096
378
- - type: mrr_at_1000
379
- value: 61.112
380
- - type: mrr_at_3
381
- value: 58.192
382
- - type: mrr_at_5
383
- value: 59.611999999999995
384
- - type: ndcg_at_1
385
- value: 50.470000000000006
386
- - type: ndcg_at_10
387
- value: 63.071999999999996
388
- - type: ndcg_at_100
389
- value: 66.964
390
- - type: ndcg_at_1000
391
- value: 67.659
392
- - type: ndcg_at_3
393
- value: 57.74399999999999
394
- - type: ndcg_at_5
395
- value: 60.367000000000004
396
- - type: precision_at_1
397
- value: 50.470000000000006
398
- - type: precision_at_10
399
- value: 10.019
400
- - type: precision_at_100
401
- value: 1.29
402
- - type: precision_at_1000
403
- value: 0.13899999999999998
404
- - type: precision_at_3
405
- value: 25.558999999999997
406
- - type: precision_at_5
407
- value: 17.467
408
- - type: recall_at_1
409
- value: 44.424
410
- - type: recall_at_10
411
- value: 77.02
412
- - type: recall_at_100
413
- value: 93.738
414
- - type: recall_at_1000
415
- value: 98.451
416
- - type: recall_at_3
417
- value: 62.888
418
- - type: recall_at_5
419
- value: 69.138
420
- - task:
421
- type: Retrieval
422
- dataset:
423
- type: BeIR/cqadupstack
424
- name: MTEB CQADupstackGisRetrieval
425
- config: default
426
- split: test
427
- revision: None
428
- metrics:
429
- - type: map_at_1
430
- value: 26.294
431
- - type: map_at_10
432
- value: 34.503
433
- - type: map_at_100
434
- value: 35.641
435
- - type: map_at_1000
436
- value: 35.724000000000004
437
- - type: map_at_3
438
- value: 31.753999999999998
439
- - type: map_at_5
440
- value: 33.190999999999995
441
- - type: mrr_at_1
442
- value: 28.362
443
- - type: mrr_at_10
444
- value: 36.53
445
- - type: mrr_at_100
446
- value: 37.541000000000004
447
- - type: mrr_at_1000
448
- value: 37.602000000000004
449
- - type: mrr_at_3
450
- value: 33.917
451
- - type: mrr_at_5
452
- value: 35.358000000000004
453
- - type: ndcg_at_1
454
- value: 28.362
455
- - type: ndcg_at_10
456
- value: 39.513999999999996
457
- - type: ndcg_at_100
458
- value: 44.815
459
- - type: ndcg_at_1000
460
- value: 46.839
461
- - type: ndcg_at_3
462
- value: 34.02
463
- - type: ndcg_at_5
464
- value: 36.522
465
- - type: precision_at_1
466
- value: 28.362
467
- - type: precision_at_10
468
- value: 6.101999999999999
469
- - type: precision_at_100
470
- value: 0.9129999999999999
471
- - type: precision_at_1000
472
- value: 0.11399999999999999
473
- - type: precision_at_3
474
- value: 14.161999999999999
475
- - type: precision_at_5
476
- value: 9.966
477
- - type: recall_at_1
478
- value: 26.294
479
- - type: recall_at_10
480
- value: 53.098
481
- - type: recall_at_100
482
- value: 76.877
483
- - type: recall_at_1000
484
- value: 91.834
485
- - type: recall_at_3
486
- value: 38.266
487
- - type: recall_at_5
488
- value: 44.287
489
- - task:
490
- type: Retrieval
491
- dataset:
492
- type: BeIR/cqadupstack
493
- name: MTEB CQADupstackMathematicaRetrieval
494
- config: default
495
- split: test
496
- revision: None
497
- metrics:
498
- - type: map_at_1
499
- value: 16.407
500
- - type: map_at_10
501
- value: 25.185999999999996
502
- - type: map_at_100
503
- value: 26.533
504
- - type: map_at_1000
505
- value: 26.657999999999998
506
- - type: map_at_3
507
- value: 22.201999999999998
508
- - type: map_at_5
509
- value: 23.923
510
- - type: mrr_at_1
511
- value: 20.522000000000002
512
- - type: mrr_at_10
513
- value: 29.522
514
- - type: mrr_at_100
515
- value: 30.644
516
- - type: mrr_at_1000
517
- value: 30.713
518
- - type: mrr_at_3
519
- value: 26.679000000000002
520
- - type: mrr_at_5
521
- value: 28.483000000000004
522
- - type: ndcg_at_1
523
- value: 20.522000000000002
524
- - type: ndcg_at_10
525
- value: 30.656
526
- - type: ndcg_at_100
527
- value: 36.864999999999995
528
- - type: ndcg_at_1000
529
- value: 39.675
530
- - type: ndcg_at_3
531
- value: 25.319000000000003
532
- - type: ndcg_at_5
533
- value: 27.992
534
- - type: precision_at_1
535
- value: 20.522000000000002
536
- - type: precision_at_10
537
- value: 5.795999999999999
538
- - type: precision_at_100
539
- value: 1.027
540
- - type: precision_at_1000
541
- value: 0.13999999999999999
542
- - type: precision_at_3
543
- value: 12.396
544
- - type: precision_at_5
545
- value: 9.328
546
- - type: recall_at_1
547
- value: 16.407
548
- - type: recall_at_10
549
- value: 43.164
550
- - type: recall_at_100
551
- value: 69.695
552
- - type: recall_at_1000
553
- value: 89.41900000000001
554
- - type: recall_at_3
555
- value: 28.634999999999998
556
- - type: recall_at_5
557
- value: 35.308
558
- - task:
559
- type: Retrieval
560
- dataset:
561
- type: BeIR/cqadupstack
562
- name: MTEB CQADupstackPhysicsRetrieval
563
- config: default
564
- split: test
565
- revision: None
566
- metrics:
567
- - type: map_at_1
568
- value: 30.473
569
- - type: map_at_10
570
- value: 41.676
571
- - type: map_at_100
572
- value: 43.120999999999995
573
- - type: map_at_1000
574
- value: 43.230000000000004
575
- - type: map_at_3
576
- value: 38.306000000000004
577
- - type: map_at_5
578
- value: 40.355999999999995
579
- - type: mrr_at_1
580
- value: 37.536
581
- - type: mrr_at_10
582
- value: 47.643
583
- - type: mrr_at_100
584
- value: 48.508
585
- - type: mrr_at_1000
586
- value: 48.551
587
- - type: mrr_at_3
588
- value: 45.348
589
- - type: mrr_at_5
590
- value: 46.744
591
- - type: ndcg_at_1
592
- value: 37.536
593
- - type: ndcg_at_10
594
- value: 47.823
595
- - type: ndcg_at_100
596
- value: 53.395
597
- - type: ndcg_at_1000
598
- value: 55.271
599
- - type: ndcg_at_3
600
- value: 42.768
601
- - type: ndcg_at_5
602
- value: 45.373000000000005
603
- - type: precision_at_1
604
- value: 37.536
605
- - type: precision_at_10
606
- value: 8.681
607
- - type: precision_at_100
608
- value: 1.34
609
- - type: precision_at_1000
610
- value: 0.165
611
- - type: precision_at_3
612
- value: 20.468
613
- - type: precision_at_5
614
- value: 14.495
615
- - type: recall_at_1
616
- value: 30.473
617
- - type: recall_at_10
618
- value: 60.092999999999996
619
- - type: recall_at_100
620
- value: 82.733
621
- - type: recall_at_1000
622
- value: 94.875
623
- - type: recall_at_3
624
- value: 45.734
625
- - type: recall_at_5
626
- value: 52.691
627
- - task:
628
- type: Retrieval
629
- dataset:
630
- type: BeIR/cqadupstack
631
- name: MTEB CQADupstackProgrammersRetrieval
632
- config: default
633
- split: test
634
- revision: None
635
- metrics:
636
- - type: map_at_1
637
- value: 29.976000000000003
638
- - type: map_at_10
639
- value: 41.097
640
- - type: map_at_100
641
- value: 42.547000000000004
642
- - type: map_at_1000
643
- value: 42.659000000000006
644
- - type: map_at_3
645
- value: 37.251
646
- - type: map_at_5
647
- value: 39.493
648
- - type: mrr_at_1
649
- value: 37.557
650
- - type: mrr_at_10
651
- value: 46.605000000000004
652
- - type: mrr_at_100
653
- value: 47.487
654
- - type: mrr_at_1000
655
- value: 47.54
656
- - type: mrr_at_3
657
- value: 43.721
658
- - type: mrr_at_5
659
- value: 45.411
660
- - type: ndcg_at_1
661
- value: 37.557
662
- - type: ndcg_at_10
663
- value: 47.449000000000005
664
- - type: ndcg_at_100
665
- value: 53.052
666
- - type: ndcg_at_1000
667
- value: 55.010999999999996
668
- - type: ndcg_at_3
669
- value: 41.439
670
- - type: ndcg_at_5
671
- value: 44.292
672
- - type: precision_at_1
673
- value: 37.557
674
- - type: precision_at_10
675
- value: 8.847
676
- - type: precision_at_100
677
- value: 1.357
678
- - type: precision_at_1000
679
- value: 0.16999999999999998
680
- - type: precision_at_3
681
- value: 20.091
682
- - type: precision_at_5
683
- value: 14.384
684
- - type: recall_at_1
685
- value: 29.976000000000003
686
- - type: recall_at_10
687
- value: 60.99099999999999
688
- - type: recall_at_100
689
- value: 84.245
690
- - type: recall_at_1000
691
- value: 96.97200000000001
692
- - type: recall_at_3
693
- value: 43.794
694
- - type: recall_at_5
695
- value: 51.778999999999996
696
- - task:
697
- type: Retrieval
698
- dataset:
699
- type: BeIR/cqadupstack
700
- name: MTEB CQADupstackRetrieval
701
- config: default
702
- split: test
703
- revision: None
704
- metrics:
705
- - type: map_at_1
706
- value: 28.099166666666665
707
- - type: map_at_10
708
- value: 38.1365
709
- - type: map_at_100
710
- value: 39.44491666666667
711
- - type: map_at_1000
712
- value: 39.55858333333334
713
- - type: map_at_3
714
- value: 35.03641666666666
715
- - type: map_at_5
716
- value: 36.79833333333334
717
- - type: mrr_at_1
718
- value: 33.39966666666667
719
- - type: mrr_at_10
720
- value: 42.42583333333333
721
- - type: mrr_at_100
722
- value: 43.28575
723
- - type: mrr_at_1000
724
- value: 43.33741666666667
725
- - type: mrr_at_3
726
- value: 39.94975
727
- - type: mrr_at_5
728
- value: 41.41633333333334
729
- - type: ndcg_at_1
730
- value: 33.39966666666667
731
- - type: ndcg_at_10
732
- value: 43.81741666666667
733
- - type: ndcg_at_100
734
- value: 49.08166666666667
735
- - type: ndcg_at_1000
736
- value: 51.121166666666674
737
- - type: ndcg_at_3
738
- value: 38.73575
739
- - type: ndcg_at_5
740
- value: 41.18158333333333
741
- - type: precision_at_1
742
- value: 33.39966666666667
743
- - type: precision_at_10
744
- value: 7.738916666666667
745
- - type: precision_at_100
746
- value: 1.2265833333333331
747
- - type: precision_at_1000
748
- value: 0.15983333333333336
749
- - type: precision_at_3
750
- value: 17.967416666666665
751
- - type: precision_at_5
752
- value: 12.78675
753
- - type: recall_at_1
754
- value: 28.099166666666665
755
- - type: recall_at_10
756
- value: 56.27049999999999
757
- - type: recall_at_100
758
- value: 78.93291666666667
759
- - type: recall_at_1000
760
- value: 92.81608333333334
761
- - type: recall_at_3
762
- value: 42.09775
763
- - type: recall_at_5
764
- value: 48.42533333333334
765
- - task:
766
- type: Retrieval
767
- dataset:
768
- type: BeIR/cqadupstack
769
- name: MTEB CQADupstackStatsRetrieval
770
- config: default
771
- split: test
772
- revision: None
773
- metrics:
774
- - type: map_at_1
775
- value: 23.663
776
- - type: map_at_10
777
- value: 30.377
778
- - type: map_at_100
779
- value: 31.426
780
- - type: map_at_1000
781
- value: 31.519000000000002
782
- - type: map_at_3
783
- value: 28.069
784
- - type: map_at_5
785
- value: 29.256999999999998
786
- - type: mrr_at_1
787
- value: 26.687
788
- - type: mrr_at_10
789
- value: 33.107
790
- - type: mrr_at_100
791
- value: 34.055
792
- - type: mrr_at_1000
793
- value: 34.117999999999995
794
- - type: mrr_at_3
795
- value: 31.058000000000003
796
- - type: mrr_at_5
797
- value: 32.14
798
- - type: ndcg_at_1
799
- value: 26.687
800
- - type: ndcg_at_10
801
- value: 34.615
802
- - type: ndcg_at_100
803
- value: 39.776
804
- - type: ndcg_at_1000
805
- value: 42.05
806
- - type: ndcg_at_3
807
- value: 30.322
808
- - type: ndcg_at_5
809
- value: 32.157000000000004
810
- - type: precision_at_1
811
- value: 26.687
812
- - type: precision_at_10
813
- value: 5.491
814
- - type: precision_at_100
815
- value: 0.877
816
- - type: precision_at_1000
817
- value: 0.11499999999999999
818
- - type: precision_at_3
819
- value: 13.139000000000001
820
- - type: precision_at_5
821
- value: 9.049
822
- - type: recall_at_1
823
- value: 23.663
824
- - type: recall_at_10
825
- value: 45.035
826
- - type: recall_at_100
827
- value: 68.554
828
- - type: recall_at_1000
829
- value: 85.077
830
- - type: recall_at_3
831
- value: 32.982
832
- - type: recall_at_5
833
- value: 37.688
834
- - task:
835
- type: Retrieval
836
- dataset:
837
- type: BeIR/cqadupstack
838
- name: MTEB CQADupstackTexRetrieval
839
- config: default
840
- split: test
841
- revision: None
842
- metrics:
843
- - type: map_at_1
844
- value: 17.403
845
- - type: map_at_10
846
- value: 25.197000000000003
847
- - type: map_at_100
848
- value: 26.355
849
- - type: map_at_1000
850
- value: 26.487
851
- - type: map_at_3
852
- value: 22.733
853
- - type: map_at_5
854
- value: 24.114
855
- - type: mrr_at_1
856
- value: 21.37
857
- - type: mrr_at_10
858
- value: 29.091
859
- - type: mrr_at_100
860
- value: 30.018
861
- - type: mrr_at_1000
862
- value: 30.096
863
- - type: mrr_at_3
864
- value: 26.887
865
- - type: mrr_at_5
866
- value: 28.157
867
- - type: ndcg_at_1
868
- value: 21.37
869
- - type: ndcg_at_10
870
- value: 30.026000000000003
871
- - type: ndcg_at_100
872
- value: 35.416
873
- - type: ndcg_at_1000
874
- value: 38.45
875
- - type: ndcg_at_3
876
- value: 25.764
877
- - type: ndcg_at_5
878
- value: 27.742
879
- - type: precision_at_1
880
- value: 21.37
881
- - type: precision_at_10
882
- value: 5.609
883
- - type: precision_at_100
884
- value: 0.9860000000000001
885
- - type: precision_at_1000
886
- value: 0.14300000000000002
887
- - type: precision_at_3
888
- value: 12.423
889
- - type: precision_at_5
890
- value: 9.009
891
- - type: recall_at_1
892
- value: 17.403
893
- - type: recall_at_10
894
- value: 40.573
895
- - type: recall_at_100
896
- value: 64.818
897
- - type: recall_at_1000
898
- value: 86.53699999999999
899
- - type: recall_at_3
900
- value: 28.493000000000002
901
- - type: recall_at_5
902
- value: 33.660000000000004
903
- - task:
904
- type: Retrieval
905
- dataset:
906
- type: BeIR/cqadupstack
907
- name: MTEB CQADupstackUnixRetrieval
908
- config: default
909
- split: test
910
- revision: None
911
- metrics:
912
- - type: map_at_1
913
- value: 28.639
914
- - type: map_at_10
915
- value: 38.951
916
- - type: map_at_100
917
- value: 40.238
918
- - type: map_at_1000
919
- value: 40.327
920
- - type: map_at_3
921
- value: 35.842
922
- - type: map_at_5
923
- value: 37.617
924
- - type: mrr_at_1
925
- value: 33.769
926
- - type: mrr_at_10
927
- value: 43.088
928
- - type: mrr_at_100
929
- value: 44.03
930
- - type: mrr_at_1000
931
- value: 44.072
932
- - type: mrr_at_3
933
- value: 40.656
934
- - type: mrr_at_5
935
- value: 42.138999999999996
936
- - type: ndcg_at_1
937
- value: 33.769
938
- - type: ndcg_at_10
939
- value: 44.676
940
- - type: ndcg_at_100
941
- value: 50.416000000000004
942
- - type: ndcg_at_1000
943
- value: 52.227999999999994
944
- - type: ndcg_at_3
945
- value: 39.494
946
- - type: ndcg_at_5
947
- value: 42.013
948
- - type: precision_at_1
949
- value: 33.769
950
- - type: precision_at_10
951
- value: 7.668
952
- - type: precision_at_100
953
- value: 1.18
954
- - type: precision_at_1000
955
- value: 0.145
956
- - type: precision_at_3
957
- value: 18.221
958
- - type: precision_at_5
959
- value: 12.966
960
- - type: recall_at_1
961
- value: 28.639
962
- - type: recall_at_10
963
- value: 57.687999999999995
964
- - type: recall_at_100
965
- value: 82.541
966
- - type: recall_at_1000
967
- value: 94.896
968
- - type: recall_at_3
969
- value: 43.651
970
- - type: recall_at_5
971
- value: 49.925999999999995
972
- - task:
973
- type: Retrieval
974
- dataset:
975
- type: BeIR/cqadupstack
976
- name: MTEB CQADupstackWebmastersRetrieval
977
- config: default
978
- split: test
979
- revision: None
980
- metrics:
981
- - type: map_at_1
982
- value: 29.57
983
- - type: map_at_10
984
- value: 40.004
985
- - type: map_at_100
986
- value: 41.75
987
- - type: map_at_1000
988
- value: 41.97
989
- - type: map_at_3
990
- value: 36.788
991
- - type: map_at_5
992
- value: 38.671
993
- - type: mrr_at_1
994
- value: 35.375
995
- - type: mrr_at_10
996
- value: 45.121
997
- - type: mrr_at_100
998
- value: 45.994
999
- - type: mrr_at_1000
1000
- value: 46.04
1001
- - type: mrr_at_3
1002
- value: 42.227
1003
- - type: mrr_at_5
1004
- value: 43.995
1005
- - type: ndcg_at_1
1006
- value: 35.375
1007
- - type: ndcg_at_10
1008
- value: 46.392
1009
- - type: ndcg_at_100
1010
- value: 52.196
1011
- - type: ndcg_at_1000
1012
- value: 54.274
1013
- - type: ndcg_at_3
1014
- value: 41.163
1015
- - type: ndcg_at_5
1016
- value: 43.813
1017
- - type: precision_at_1
1018
- value: 35.375
1019
- - type: precision_at_10
1020
- value: 8.676
1021
- - type: precision_at_100
1022
- value: 1.678
1023
- - type: precision_at_1000
1024
- value: 0.253
1025
- - type: precision_at_3
1026
- value: 19.104
1027
- - type: precision_at_5
1028
- value: 13.913
1029
- - type: recall_at_1
1030
- value: 29.57
1031
- - type: recall_at_10
1032
- value: 58.779
1033
- - type: recall_at_100
1034
- value: 83.337
1035
- - type: recall_at_1000
1036
- value: 95.979
1037
- - type: recall_at_3
1038
- value: 44.005
1039
- - type: recall_at_5
1040
- value: 50.975
1041
- - task:
1042
- type: Retrieval
1043
- dataset:
1044
- type: BeIR/cqadupstack
1045
- name: MTEB CQADupstackWordpressRetrieval
1046
- config: default
1047
- split: test
1048
- revision: None
1049
- metrics:
1050
- - type: map_at_1
1051
- value: 20.832
1052
- - type: map_at_10
1053
- value: 29.733999999999998
1054
- - type: map_at_100
1055
- value: 30.727
1056
- - type: map_at_1000
1057
- value: 30.843999999999998
1058
- - type: map_at_3
1059
- value: 26.834999999999997
1060
- - type: map_at_5
1061
- value: 28.555999999999997
1062
- - type: mrr_at_1
1063
- value: 22.921
1064
- - type: mrr_at_10
1065
- value: 31.791999999999998
1066
- - type: mrr_at_100
1067
- value: 32.666000000000004
1068
- - type: mrr_at_1000
1069
- value: 32.751999999999995
1070
- - type: mrr_at_3
1071
- value: 29.144
1072
- - type: mrr_at_5
1073
- value: 30.622
1074
- - type: ndcg_at_1
1075
- value: 22.921
1076
- - type: ndcg_at_10
1077
- value: 34.915
1078
- - type: ndcg_at_100
1079
- value: 39.744
1080
- - type: ndcg_at_1000
1081
- value: 42.407000000000004
1082
- - type: ndcg_at_3
1083
- value: 29.421000000000003
1084
- - type: ndcg_at_5
1085
- value: 32.211
1086
- - type: precision_at_1
1087
- value: 22.921
1088
- - type: precision_at_10
1089
- value: 5.675
1090
- - type: precision_at_100
1091
- value: 0.872
1092
- - type: precision_at_1000
1093
- value: 0.121
1094
- - type: precision_at_3
1095
- value: 12.753999999999998
1096
- - type: precision_at_5
1097
- value: 9.353
1098
- - type: recall_at_1
1099
- value: 20.832
1100
- - type: recall_at_10
1101
- value: 48.795
1102
- - type: recall_at_100
1103
- value: 70.703
1104
- - type: recall_at_1000
1105
- value: 90.187
1106
- - type: recall_at_3
1107
- value: 34.455000000000005
1108
- - type: recall_at_5
1109
- value: 40.967
1110
- - task:
1111
- type: Retrieval
1112
- dataset:
1113
- type: climate-fever
1114
- name: MTEB ClimateFEVER
1115
- config: default
1116
- split: test
1117
- revision: None
1118
- metrics:
1119
- - type: map_at_1
1120
- value: 10.334
1121
- - type: map_at_10
1122
- value: 19.009999999999998
1123
- - type: map_at_100
1124
- value: 21.129
1125
- - type: map_at_1000
1126
- value: 21.328
1127
- - type: map_at_3
1128
- value: 15.152
1129
- - type: map_at_5
1130
- value: 17.084
1131
- - type: mrr_at_1
1132
- value: 23.453
1133
- - type: mrr_at_10
1134
- value: 36.099
1135
- - type: mrr_at_100
1136
- value: 37.069
1137
- - type: mrr_at_1000
1138
- value: 37.104
1139
- - type: mrr_at_3
1140
- value: 32.096000000000004
1141
- - type: mrr_at_5
1142
- value: 34.451
1143
- - type: ndcg_at_1
1144
- value: 23.453
1145
- - type: ndcg_at_10
1146
- value: 27.739000000000004
1147
- - type: ndcg_at_100
1148
- value: 35.836
1149
- - type: ndcg_at_1000
1150
- value: 39.242
1151
- - type: ndcg_at_3
1152
- value: 21.263
1153
- - type: ndcg_at_5
1154
- value: 23.677
1155
- - type: precision_at_1
1156
- value: 23.453
1157
- - type: precision_at_10
1158
- value: 9.199
1159
- - type: precision_at_100
1160
- value: 1.791
1161
- - type: precision_at_1000
1162
- value: 0.242
1163
- - type: precision_at_3
1164
- value: 16.2
1165
- - type: precision_at_5
1166
- value: 13.147
1167
- - type: recall_at_1
1168
- value: 10.334
1169
- - type: recall_at_10
1170
- value: 35.177
1171
- - type: recall_at_100
1172
- value: 63.009
1173
- - type: recall_at_1000
1174
- value: 81.938
1175
- - type: recall_at_3
1176
- value: 19.914
1177
- - type: recall_at_5
1178
- value: 26.077
1179
- - task:
1180
- type: Retrieval
1181
- dataset:
1182
- type: dbpedia-entity
1183
- name: MTEB DBPedia
1184
- config: default
1185
- split: test
1186
- revision: None
1187
- metrics:
1188
- - type: map_at_1
1189
- value: 8.212
1190
- - type: map_at_10
1191
- value: 17.386
1192
- - type: map_at_100
1193
- value: 24.234
1194
- - type: map_at_1000
1195
- value: 25.724999999999998
1196
- - type: map_at_3
1197
- value: 12.727
1198
- - type: map_at_5
1199
- value: 14.785
1200
- - type: mrr_at_1
1201
- value: 59.25
1202
- - type: mrr_at_10
1203
- value: 68.687
1204
- - type: mrr_at_100
1205
- value: 69.133
1206
- - type: mrr_at_1000
1207
- value: 69.14099999999999
1208
- - type: mrr_at_3
1209
- value: 66.917
1210
- - type: mrr_at_5
1211
- value: 67.742
1212
- - type: ndcg_at_1
1213
- value: 48.625
1214
- - type: ndcg_at_10
1215
- value: 36.675999999999995
1216
- - type: ndcg_at_100
1217
- value: 41.543
1218
- - type: ndcg_at_1000
1219
- value: 49.241
1220
- - type: ndcg_at_3
1221
- value: 41.373
1222
- - type: ndcg_at_5
1223
- value: 38.707
1224
- - type: precision_at_1
1225
- value: 59.25
1226
- - type: precision_at_10
1227
- value: 28.525
1228
- - type: precision_at_100
1229
- value: 9.027000000000001
1230
- - type: precision_at_1000
1231
- value: 1.8339999999999999
1232
- - type: precision_at_3
1233
- value: 44.833
1234
- - type: precision_at_5
1235
- value: 37.35
1236
- - type: recall_at_1
1237
- value: 8.212
1238
- - type: recall_at_10
1239
- value: 23.188
1240
- - type: recall_at_100
1241
- value: 48.613
1242
- - type: recall_at_1000
1243
- value: 73.093
1244
- - type: recall_at_3
1245
- value: 14.419
1246
- - type: recall_at_5
1247
- value: 17.798
1248
- - task:
1249
- type: Classification
1250
- dataset:
1251
- type: mteb/emotion
1252
- name: MTEB EmotionClassification
1253
- config: default
1254
- split: test
1255
- revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
1256
- metrics:
1257
- - type: accuracy
1258
- value: 52.725
1259
- - type: f1
1260
- value: 46.50743309855908
1261
- - task:
1262
- type: Retrieval
1263
- dataset:
1264
- type: fever
1265
- name: MTEB FEVER
1266
- config: default
1267
- split: test
1268
- revision: None
1269
- metrics:
1270
- - type: map_at_1
1271
- value: 55.086
1272
- - type: map_at_10
1273
- value: 66.914
1274
- - type: map_at_100
1275
- value: 67.321
1276
- - type: map_at_1000
1277
- value: 67.341
1278
- - type: map_at_3
1279
- value: 64.75800000000001
1280
- - type: map_at_5
1281
- value: 66.189
1282
- - type: mrr_at_1
1283
- value: 59.28600000000001
1284
- - type: mrr_at_10
1285
- value: 71.005
1286
- - type: mrr_at_100
1287
- value: 71.304
1288
- - type: mrr_at_1000
1289
- value: 71.313
1290
- - type: mrr_at_3
1291
- value: 69.037
1292
- - type: mrr_at_5
1293
- value: 70.35
1294
- - type: ndcg_at_1
1295
- value: 59.28600000000001
1296
- - type: ndcg_at_10
1297
- value: 72.695
1298
- - type: ndcg_at_100
1299
- value: 74.432
1300
- - type: ndcg_at_1000
1301
- value: 74.868
1302
- - type: ndcg_at_3
1303
- value: 68.72200000000001
1304
- - type: ndcg_at_5
1305
- value: 71.081
1306
- - type: precision_at_1
1307
- value: 59.28600000000001
1308
- - type: precision_at_10
1309
- value: 9.499
1310
- - type: precision_at_100
1311
- value: 1.052
1312
- - type: precision_at_1000
1313
- value: 0.11100000000000002
1314
- - type: precision_at_3
1315
- value: 27.503
1316
- - type: precision_at_5
1317
- value: 17.854999999999997
1318
- - type: recall_at_1
1319
- value: 55.086
1320
- - type: recall_at_10
1321
- value: 86.453
1322
- - type: recall_at_100
1323
- value: 94.028
1324
- - type: recall_at_1000
1325
- value: 97.052
1326
- - type: recall_at_3
1327
- value: 75.821
1328
- - type: recall_at_5
1329
- value: 81.6
1330
- - task:
1331
- type: Retrieval
1332
- dataset:
1333
- type: fiqa
1334
- name: MTEB FiQA2018
1335
- config: default
1336
- split: test
1337
- revision: None
1338
- metrics:
1339
- - type: map_at_1
1340
- value: 22.262999999999998
1341
- - type: map_at_10
1342
- value: 37.488
1343
- - type: map_at_100
1344
- value: 39.498
1345
- - type: map_at_1000
1346
- value: 39.687
1347
- - type: map_at_3
1348
- value: 32.529
1349
- - type: map_at_5
1350
- value: 35.455
1351
- - type: mrr_at_1
1352
- value: 44.907000000000004
1353
- - type: mrr_at_10
1354
- value: 53.239000000000004
1355
- - type: mrr_at_100
1356
- value: 54.086
1357
- - type: mrr_at_1000
1358
- value: 54.122
1359
- - type: mrr_at_3
1360
- value: 51.235
1361
- - type: mrr_at_5
1362
- value: 52.415
1363
- - type: ndcg_at_1
1364
- value: 44.907000000000004
1365
- - type: ndcg_at_10
1366
- value: 45.446
1367
- - type: ndcg_at_100
1368
- value: 52.429
1369
- - type: ndcg_at_1000
1370
- value: 55.169000000000004
1371
- - type: ndcg_at_3
1372
- value: 41.882000000000005
1373
- - type: ndcg_at_5
1374
- value: 43.178
1375
- - type: precision_at_1
1376
- value: 44.907000000000004
1377
- - type: precision_at_10
1378
- value: 12.931999999999999
1379
- - type: precision_at_100
1380
- value: 2.025
1381
- - type: precision_at_1000
1382
- value: 0.248
1383
- - type: precision_at_3
1384
- value: 28.652
1385
- - type: precision_at_5
1386
- value: 21.204
1387
- - type: recall_at_1
1388
- value: 22.262999999999998
1389
- - type: recall_at_10
1390
- value: 52.447
1391
- - type: recall_at_100
1392
- value: 78.045
1393
- - type: recall_at_1000
1394
- value: 94.419
1395
- - type: recall_at_3
1396
- value: 38.064
1397
- - type: recall_at_5
1398
- value: 44.769
1399
- - task:
1400
- type: Retrieval
1401
- dataset:
1402
- type: hotpotqa
1403
- name: MTEB HotpotQA
1404
- config: default
1405
- split: test
1406
- revision: None
1407
- metrics:
1408
- - type: map_at_1
1409
- value: 32.519
1410
- - type: map_at_10
1411
- value: 45.831
1412
- - type: map_at_100
1413
- value: 46.815
1414
- - type: map_at_1000
1415
- value: 46.899
1416
- - type: map_at_3
1417
- value: 42.836
1418
- - type: map_at_5
1419
- value: 44.65
1420
- - type: mrr_at_1
1421
- value: 65.037
1422
- - type: mrr_at_10
1423
- value: 72.16
1424
- - type: mrr_at_100
1425
- value: 72.51100000000001
1426
- - type: mrr_at_1000
1427
- value: 72.53
1428
- - type: mrr_at_3
1429
- value: 70.682
1430
- - type: mrr_at_5
1431
- value: 71.54599999999999
1432
- - type: ndcg_at_1
1433
- value: 65.037
1434
- - type: ndcg_at_10
1435
- value: 55.17999999999999
1436
- - type: ndcg_at_100
1437
- value: 58.888
1438
- - type: ndcg_at_1000
1439
- value: 60.648
1440
- - type: ndcg_at_3
1441
- value: 50.501
1442
- - type: ndcg_at_5
1443
- value: 52.977
1444
- - type: precision_at_1
1445
- value: 65.037
1446
- - type: precision_at_10
1447
- value: 11.530999999999999
1448
- - type: precision_at_100
1449
- value: 1.4460000000000002
1450
- - type: precision_at_1000
1451
- value: 0.168
1452
- - type: precision_at_3
1453
- value: 31.483
1454
- - type: precision_at_5
1455
- value: 20.845
1456
- - type: recall_at_1
1457
- value: 32.519
1458
- - type: recall_at_10
1459
- value: 57.657000000000004
1460
- - type: recall_at_100
1461
- value: 72.30199999999999
1462
- - type: recall_at_1000
1463
- value: 84.024
1464
- - type: recall_at_3
1465
- value: 47.225
1466
- - type: recall_at_5
1467
- value: 52.113
1468
- - task:
1469
- type: Classification
1470
- dataset:
1471
- type: mteb/imdb
1472
- name: MTEB ImdbClassification
1473
- config: default
1474
- split: test
1475
- revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
1476
- metrics:
1477
- - type: accuracy
1478
- value: 88.3168
1479
- - type: ap
1480
- value: 83.80165516037135
1481
- - type: f1
1482
- value: 88.29942471066407
1483
- - task:
1484
- type: Retrieval
1485
- dataset:
1486
- type: msmarco
1487
- name: MTEB MSMARCO
1488
- config: default
1489
- split: dev
1490
- revision: None
1491
- metrics:
1492
- - type: map_at_1
1493
- value: 20.724999999999998
1494
- - type: map_at_10
1495
- value: 32.736
1496
- - type: map_at_100
1497
- value: 33.938
1498
- - type: map_at_1000
1499
- value: 33.991
1500
- - type: map_at_3
1501
- value: 28.788000000000004
1502
- - type: map_at_5
1503
- value: 31.016
1504
- - type: mrr_at_1
1505
- value: 21.361
1506
- - type: mrr_at_10
1507
- value: 33.323
1508
- - type: mrr_at_100
1509
- value: 34.471000000000004
1510
- - type: mrr_at_1000
1511
- value: 34.518
1512
- - type: mrr_at_3
1513
- value: 29.453000000000003
1514
- - type: mrr_at_5
1515
- value: 31.629
1516
- - type: ndcg_at_1
1517
- value: 21.361
1518
- - type: ndcg_at_10
1519
- value: 39.649
1520
- - type: ndcg_at_100
1521
- value: 45.481
1522
- - type: ndcg_at_1000
1523
- value: 46.775
1524
- - type: ndcg_at_3
1525
- value: 31.594
1526
- - type: ndcg_at_5
1527
- value: 35.543
1528
- - type: precision_at_1
1529
- value: 21.361
1530
- - type: precision_at_10
1531
- value: 6.3740000000000006
1532
- - type: precision_at_100
1533
- value: 0.931
1534
- - type: precision_at_1000
1535
- value: 0.104
1536
- - type: precision_at_3
1537
- value: 13.514999999999999
1538
- - type: precision_at_5
1539
- value: 10.100000000000001
1540
- - type: recall_at_1
1541
- value: 20.724999999999998
1542
- - type: recall_at_10
1543
- value: 61.034
1544
- - type: recall_at_100
1545
- value: 88.062
1546
- - type: recall_at_1000
1547
- value: 97.86399999999999
1548
- - type: recall_at_3
1549
- value: 39.072
1550
- - type: recall_at_5
1551
- value: 48.53
1552
- - task:
1553
- type: Classification
1554
- dataset:
1555
- type: mteb/mtop_domain
1556
- name: MTEB MTOPDomainClassification (en)
1557
- config: en
1558
- split: test
1559
- revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1560
- metrics:
1561
- - type: accuracy
1562
- value: 93.8919288645691
1563
- - type: f1
1564
- value: 93.57059586398059
1565
- - task:
1566
- type: Classification
1567
- dataset:
1568
- type: mteb/mtop_intent
1569
- name: MTEB MTOPIntentClassification (en)
1570
- config: en
1571
- split: test
1572
- revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1573
- metrics:
1574
- - type: accuracy
1575
- value: 67.97993616051072
1576
- - type: f1
1577
- value: 48.244319183606535
1578
- - task:
1579
- type: Classification
1580
- dataset:
1581
- type: mteb/amazon_massive_intent
1582
- name: MTEB MassiveIntentClassification (en)
1583
- config: en
1584
- split: test
1585
- revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1586
- metrics:
1587
- - type: accuracy
1588
- value: 68.90047074646941
1589
- - type: f1
1590
- value: 66.48999056063725
1591
- - task:
1592
- type: Classification
1593
- dataset:
1594
- type: mteb/amazon_massive_scenario
1595
- name: MTEB MassiveScenarioClassification (en)
1596
- config: en
1597
- split: test
1598
- revision: 7d571f92784cd94a019292a1f45445077d0ef634
1599
- metrics:
1600
- - type: accuracy
1601
- value: 73.34566240753195
1602
- - type: f1
1603
- value: 73.54164154290658
1604
- - task:
1605
- type: Clustering
1606
- dataset:
1607
- type: mteb/medrxiv-clustering-p2p
1608
- name: MTEB MedrxivClusteringP2P
1609
- config: default
1610
- split: test
1611
- revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
1612
- metrics:
1613
- - type: v_measure
1614
- value: 34.21866934757011
1615
- - task:
1616
- type: Clustering
1617
- dataset:
1618
- type: mteb/medrxiv-clustering-s2s
1619
- name: MTEB MedrxivClusteringS2S
1620
- config: default
1621
- split: test
1622
- revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
1623
- metrics:
1624
- - type: v_measure
1625
- value: 32.000936217235534
1626
- - task:
1627
- type: Reranking
1628
- dataset:
1629
- type: mteb/mind_small
1630
- name: MTEB MindSmallReranking
1631
- config: default
1632
- split: test
1633
- revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
1634
- metrics:
1635
- - type: map
1636
- value: 31.68189362520352
1637
- - type: mrr
1638
- value: 32.69603637784303
1639
- - task:
1640
- type: Retrieval
1641
- dataset:
1642
- type: nfcorpus
1643
- name: MTEB NFCorpus
1644
- config: default
1645
- split: test
1646
- revision: None
1647
- metrics:
1648
- - type: map_at_1
1649
- value: 6.078
1650
- - type: map_at_10
1651
- value: 12.671
1652
- - type: map_at_100
1653
- value: 16.291
1654
- - type: map_at_1000
1655
- value: 17.855999999999998
1656
- - type: map_at_3
1657
- value: 9.610000000000001
1658
- - type: map_at_5
1659
- value: 11.152
1660
- - type: mrr_at_1
1661
- value: 43.963
1662
- - type: mrr_at_10
1663
- value: 53.173
1664
- - type: mrr_at_100
1665
- value: 53.718999999999994
1666
- - type: mrr_at_1000
1667
- value: 53.756
1668
- - type: mrr_at_3
1669
- value: 50.980000000000004
1670
- - type: mrr_at_5
1671
- value: 52.42
1672
- - type: ndcg_at_1
1673
- value: 42.415000000000006
1674
- - type: ndcg_at_10
1675
- value: 34.086
1676
- - type: ndcg_at_100
1677
- value: 32.545
1678
- - type: ndcg_at_1000
1679
- value: 41.144999999999996
1680
- - type: ndcg_at_3
1681
- value: 39.434999999999995
1682
- - type: ndcg_at_5
1683
- value: 37.888
1684
- - type: precision_at_1
1685
- value: 43.653
1686
- - type: precision_at_10
1687
- value: 25.014999999999997
1688
- - type: precision_at_100
1689
- value: 8.594
1690
- - type: precision_at_1000
1691
- value: 2.169
1692
- - type: precision_at_3
1693
- value: 37.049
1694
- - type: precision_at_5
1695
- value: 33.065
1696
- - type: recall_at_1
1697
- value: 6.078
1698
- - type: recall_at_10
1699
- value: 16.17
1700
- - type: recall_at_100
1701
- value: 34.512
1702
- - type: recall_at_1000
1703
- value: 65.447
1704
- - type: recall_at_3
1705
- value: 10.706
1706
- - type: recall_at_5
1707
- value: 13.158
1708
- - task:
1709
- type: Retrieval
1710
- dataset:
1711
- type: nq
1712
- name: MTEB NQ
1713
- config: default
1714
- split: test
1715
- revision: None
1716
- metrics:
1717
- - type: map_at_1
1718
- value: 27.378000000000004
1719
- - type: map_at_10
1720
- value: 42.178
1721
- - type: map_at_100
1722
- value: 43.32
1723
- - type: map_at_1000
1724
- value: 43.358000000000004
1725
- - type: map_at_3
1726
- value: 37.474000000000004
1727
- - type: map_at_5
1728
- value: 40.333000000000006
1729
- - type: mrr_at_1
1730
- value: 30.823
1731
- - type: mrr_at_10
1732
- value: 44.626
1733
- - type: mrr_at_100
1734
- value: 45.494
1735
- - type: mrr_at_1000
1736
- value: 45.519
1737
- - type: mrr_at_3
1738
- value: 40.585
1739
- - type: mrr_at_5
1740
- value: 43.146
1741
- - type: ndcg_at_1
1742
- value: 30.794
1743
- - type: ndcg_at_10
1744
- value: 50.099000000000004
1745
- - type: ndcg_at_100
1746
- value: 54.900999999999996
1747
- - type: ndcg_at_1000
1748
- value: 55.69499999999999
1749
- - type: ndcg_at_3
1750
- value: 41.238
1751
- - type: ndcg_at_5
1752
- value: 46.081
1753
- - type: precision_at_1
1754
- value: 30.794
1755
- - type: precision_at_10
1756
- value: 8.549
1757
- - type: precision_at_100
1758
- value: 1.124
1759
- - type: precision_at_1000
1760
- value: 0.12
1761
- - type: precision_at_3
1762
- value: 18.926000000000002
1763
- - type: precision_at_5
1764
- value: 14.16
1765
- - type: recall_at_1
1766
- value: 27.378000000000004
1767
- - type: recall_at_10
1768
- value: 71.842
1769
- - type: recall_at_100
1770
- value: 92.565
1771
- - type: recall_at_1000
1772
- value: 98.402
1773
- - type: recall_at_3
1774
- value: 49.053999999999995
1775
- - type: recall_at_5
1776
- value: 60.207
1777
- - task:
1778
- type: Retrieval
1779
- dataset:
1780
- type: quora
1781
- name: MTEB QuoraRetrieval
1782
- config: default
1783
- split: test
1784
- revision: None
1785
- metrics:
1786
- - type: map_at_1
1787
- value: 70.557
1788
- - type: map_at_10
1789
- value: 84.729
1790
- - type: map_at_100
1791
- value: 85.369
1792
- - type: map_at_1000
1793
- value: 85.382
1794
- - type: map_at_3
1795
- value: 81.72
1796
- - type: map_at_5
1797
- value: 83.613
1798
- - type: mrr_at_1
1799
- value: 81.3
1800
- - type: mrr_at_10
1801
- value: 87.488
1802
- - type: mrr_at_100
1803
- value: 87.588
1804
- - type: mrr_at_1000
1805
- value: 87.589
1806
- - type: mrr_at_3
1807
- value: 86.53
1808
- - type: mrr_at_5
1809
- value: 87.18599999999999
1810
- - type: ndcg_at_1
1811
- value: 81.28999999999999
1812
- - type: ndcg_at_10
1813
- value: 88.442
1814
- - type: ndcg_at_100
1815
- value: 89.637
1816
- - type: ndcg_at_1000
1817
- value: 89.70700000000001
1818
- - type: ndcg_at_3
1819
- value: 85.55199999999999
1820
- - type: ndcg_at_5
1821
- value: 87.154
1822
- - type: precision_at_1
1823
- value: 81.28999999999999
1824
- - type: precision_at_10
1825
- value: 13.489999999999998
1826
- - type: precision_at_100
1827
- value: 1.54
1828
- - type: precision_at_1000
1829
- value: 0.157
1830
- - type: precision_at_3
1831
- value: 37.553
1832
- - type: precision_at_5
1833
- value: 24.708
1834
- - type: recall_at_1
1835
- value: 70.557
1836
- - type: recall_at_10
1837
- value: 95.645
1838
- - type: recall_at_100
1839
- value: 99.693
1840
- - type: recall_at_1000
1841
- value: 99.995
1842
- - type: recall_at_3
1843
- value: 87.359
1844
- - type: recall_at_5
1845
- value: 91.89699999999999
1846
- - task:
1847
- type: Clustering
1848
- dataset:
1849
- type: mteb/reddit-clustering
1850
- name: MTEB RedditClustering
1851
- config: default
1852
- split: test
1853
- revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
1854
- metrics:
1855
- - type: v_measure
1856
- value: 63.65060114776209
1857
- - task:
1858
- type: Clustering
1859
- dataset:
1860
- type: mteb/reddit-clustering-p2p
1861
- name: MTEB RedditClusteringP2P
1862
- config: default
1863
- split: test
1864
- revision: 282350215ef01743dc01b456c7f5241fa8937f16
1865
- metrics:
1866
- - type: v_measure
1867
- value: 64.63271250680617
1868
- - task:
1869
- type: Retrieval
1870
- dataset:
1871
- type: scidocs
1872
- name: MTEB SCIDOCS
1873
- config: default
1874
- split: test
1875
- revision: None
1876
- metrics:
1877
- - type: map_at_1
1878
- value: 4.263
1879
- - type: map_at_10
1880
- value: 10.801
1881
- - type: map_at_100
1882
- value: 12.888
1883
- - type: map_at_1000
1884
- value: 13.224
1885
- - type: map_at_3
1886
- value: 7.362
1887
- - type: map_at_5
1888
- value: 9.149000000000001
1889
- - type: mrr_at_1
1890
- value: 21
1891
- - type: mrr_at_10
1892
- value: 31.416
1893
- - type: mrr_at_100
1894
- value: 32.513
1895
- - type: mrr_at_1000
1896
- value: 32.58
1897
- - type: mrr_at_3
1898
- value: 28.116999999999997
1899
- - type: mrr_at_5
1900
- value: 29.976999999999997
1901
- - type: ndcg_at_1
1902
- value: 21
1903
- - type: ndcg_at_10
1904
- value: 18.551000000000002
1905
- - type: ndcg_at_100
1906
- value: 26.657999999999998
1907
- - type: ndcg_at_1000
1908
- value: 32.485
1909
- - type: ndcg_at_3
1910
- value: 16.834
1911
- - type: ndcg_at_5
1912
- value: 15.204999999999998
1913
- - type: precision_at_1
1914
- value: 21
1915
- - type: precision_at_10
1916
- value: 9.84
1917
- - type: precision_at_100
1918
- value: 2.16
1919
- - type: precision_at_1000
1920
- value: 0.35500000000000004
1921
- - type: precision_at_3
1922
- value: 15.667
1923
- - type: precision_at_5
1924
- value: 13.62
1925
- - type: recall_at_1
1926
- value: 4.263
1927
- - type: recall_at_10
1928
- value: 19.922
1929
- - type: recall_at_100
1930
- value: 43.808
1931
- - type: recall_at_1000
1932
- value: 72.14500000000001
1933
- - type: recall_at_3
1934
- value: 9.493
1935
- - type: recall_at_5
1936
- value: 13.767999999999999
1937
- - task:
1938
- type: STS
1939
- dataset:
1940
- type: mteb/sickr-sts
1941
- name: MTEB SICK-R
1942
- config: default
1943
- split: test
1944
- revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
1945
- metrics:
1946
- - type: cos_sim_spearman
1947
- value: 81.27446313317233
1948
- - task:
1949
- type: STS
1950
- dataset:
1951
- type: mteb/sts12-sts
1952
- name: MTEB STS12
1953
- config: default
1954
- split: test
1955
- revision: a0d554a64d88156834ff5ae9920b964011b16384
1956
- metrics:
1957
- - type: cos_sim_spearman
1958
- value: 76.27963301217527
1959
- - task:
1960
- type: STS
1961
- dataset:
1962
- type: mteb/sts13-sts
1963
- name: MTEB STS13
1964
- config: default
1965
- split: test
1966
- revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
1967
- metrics:
1968
- - type: cos_sim_spearman
1969
- value: 88.18495048450949
1970
- - task:
1971
- type: STS
1972
- dataset:
1973
- type: mteb/sts14-sts
1974
- name: MTEB STS14
1975
- config: default
1976
- split: test
1977
- revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
1978
- metrics:
1979
- - type: cos_sim_spearman
1980
- value: 81.91982338692046
1981
- - task:
1982
- type: STS
1983
- dataset:
1984
- type: mteb/sts15-sts
1985
- name: MTEB STS15
1986
- config: default
1987
- split: test
1988
- revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
1989
- metrics:
1990
- - type: cos_sim_spearman
1991
- value: 89.00896818385291
1992
- - task:
1993
- type: STS
1994
- dataset:
1995
- type: mteb/sts16-sts
1996
- name: MTEB STS16
1997
- config: default
1998
- split: test
1999
- revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
2000
- metrics:
2001
- - type: cos_sim_spearman
2002
- value: 85.48814644586132
2003
- - task:
2004
- type: STS
2005
- dataset:
2006
- type: mteb/sts17-crosslingual-sts
2007
- name: MTEB STS17 (en-en)
2008
- config: en-en
2009
- split: test
2010
- revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2011
- metrics:
2012
- - type: cos_sim_spearman
2013
- value: 90.30116926966582
2014
- - task:
2015
- type: STS
2016
- dataset:
2017
- type: mteb/sts22-crosslingual-sts
2018
- name: MTEB STS22 (en)
2019
- config: en
2020
- split: test
2021
- revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
2022
- metrics:
2023
- - type: cos_sim_spearman
2024
- value: 67.74132963032342
2025
- - task:
2026
- type: STS
2027
- dataset:
2028
- type: mteb/stsbenchmark-sts
2029
- name: MTEB STSBenchmark
2030
- config: default
2031
- split: test
2032
- revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
2033
- metrics:
2034
- - type: cos_sim_spearman
2035
- value: 86.87741355780479
2036
- - task:
2037
- type: Reranking
2038
- dataset:
2039
- type: mteb/scidocs-reranking
2040
- name: MTEB SciDocsRR
2041
- config: default
2042
- split: test
2043
- revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
2044
- metrics:
2045
- - type: map
2046
- value: 82.0019012295875
2047
- - type: mrr
2048
- value: 94.70267024188593
2049
- - task:
2050
- type: Retrieval
2051
- dataset:
2052
- type: scifact
2053
- name: MTEB SciFact
2054
- config: default
2055
- split: test
2056
- revision: None
2057
- metrics:
2058
- - type: map_at_1
2059
- value: 50.05
2060
- - type: map_at_10
2061
- value: 59.36
2062
- - type: map_at_100
2063
- value: 59.967999999999996
2064
- - type: map_at_1000
2065
- value: 60.023
2066
- - type: map_at_3
2067
- value: 56.515
2068
- - type: map_at_5
2069
- value: 58.272999999999996
2070
- - type: mrr_at_1
2071
- value: 53
2072
- - type: mrr_at_10
2073
- value: 61.102000000000004
2074
- - type: mrr_at_100
2075
- value: 61.476
2076
- - type: mrr_at_1000
2077
- value: 61.523
2078
- - type: mrr_at_3
2079
- value: 58.778
2080
- - type: mrr_at_5
2081
- value: 60.128
2082
- - type: ndcg_at_1
2083
- value: 53
2084
- - type: ndcg_at_10
2085
- value: 64.43100000000001
2086
- - type: ndcg_at_100
2087
- value: 66.73599999999999
2088
- - type: ndcg_at_1000
2089
- value: 68.027
2090
- - type: ndcg_at_3
2091
- value: 59.279
2092
- - type: ndcg_at_5
2093
- value: 61.888
2094
- - type: precision_at_1
2095
- value: 53
2096
- - type: precision_at_10
2097
- value: 8.767
2098
- - type: precision_at_100
2099
- value: 1.01
2100
- - type: precision_at_1000
2101
- value: 0.11100000000000002
2102
- - type: precision_at_3
2103
- value: 23.444000000000003
2104
- - type: precision_at_5
2105
- value: 15.667
2106
- - type: recall_at_1
2107
- value: 50.05
2108
- - type: recall_at_10
2109
- value: 78.511
2110
- - type: recall_at_100
2111
- value: 88.5
2112
- - type: recall_at_1000
2113
- value: 98.333
2114
- - type: recall_at_3
2115
- value: 64.117
2116
- - type: recall_at_5
2117
- value: 70.867
2118
- - task:
2119
- type: PairClassification
2120
- dataset:
2121
- type: mteb/sprintduplicatequestions-pairclassification
2122
- name: MTEB SprintDuplicateQuestions
2123
- config: default
2124
- split: test
2125
- revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
2126
- metrics:
2127
- - type: cos_sim_accuracy
2128
- value: 99.72178217821782
2129
- - type: cos_sim_ap
2130
- value: 93.0728601593541
2131
- - type: cos_sim_f1
2132
- value: 85.6727976766699
2133
- - type: cos_sim_precision
2134
- value: 83.02063789868667
2135
- - type: cos_sim_recall
2136
- value: 88.5
2137
- - type: dot_accuracy
2138
- value: 99.72178217821782
2139
- - type: dot_ap
2140
- value: 93.07287396168348
2141
- - type: dot_f1
2142
- value: 85.6727976766699
2143
- - type: dot_precision
2144
- value: 83.02063789868667
2145
- - type: dot_recall
2146
- value: 88.5
2147
- - type: euclidean_accuracy
2148
- value: 99.72178217821782
2149
- - type: euclidean_ap
2150
- value: 93.07285657982895
2151
- - type: euclidean_f1
2152
- value: 85.6727976766699
2153
- - type: euclidean_precision
2154
- value: 83.02063789868667
2155
- - type: euclidean_recall
2156
- value: 88.5
2157
- - type: manhattan_accuracy
2158
- value: 99.72475247524753
2159
- - type: manhattan_ap
2160
- value: 93.02792973059809
2161
- - type: manhattan_f1
2162
- value: 85.7727737973388
2163
- - type: manhattan_precision
2164
- value: 87.84067085953879
2165
- - type: manhattan_recall
2166
- value: 83.8
2167
- - type: max_accuracy
2168
- value: 99.72475247524753
2169
- - type: max_ap
2170
- value: 93.07287396168348
2171
- - type: max_f1
2172
- value: 85.7727737973388
2173
- - task:
2174
- type: Clustering
2175
- dataset:
2176
- type: mteb/stackexchange-clustering
2177
- name: MTEB StackExchangeClustering
2178
- config: default
2179
- split: test
2180
- revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
2181
- metrics:
2182
- - type: v_measure
2183
- value: 68.77583615550819
2184
- - task:
2185
- type: Clustering
2186
- dataset:
2187
- type: mteb/stackexchange-clustering-p2p
2188
- name: MTEB StackExchangeClusteringP2P
2189
- config: default
2190
- split: test
2191
- revision: 815ca46b2622cec33ccafc3735d572c266efdb44
2192
- metrics:
2193
- - type: v_measure
2194
- value: 36.151636938606956
2195
- - task:
2196
- type: Reranking
2197
- dataset:
2198
- type: mteb/stackoverflowdupquestions-reranking
2199
- name: MTEB StackOverflowDupQuestions
2200
- config: default
2201
- split: test
2202
- revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
2203
- metrics:
2204
- - type: map
2205
- value: 52.16607939471187
2206
- - type: mrr
2207
- value: 52.95172046091163
2208
- - task:
2209
- type: Summarization
2210
- dataset:
2211
- type: mteb/summeval
2212
- name: MTEB SummEval
2213
- config: default
2214
- split: test
2215
- revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
2216
- metrics:
2217
- - type: cos_sim_pearson
2218
- value: 31.314646669495666
2219
- - type: cos_sim_spearman
2220
- value: 31.83562491439455
2221
- - type: dot_pearson
2222
- value: 31.314590842874157
2223
- - type: dot_spearman
2224
- value: 31.83363065810437
2225
- - task:
2226
- type: Retrieval
2227
- dataset:
2228
- type: trec-covid
2229
- name: MTEB TRECCOVID
2230
- config: default
2231
- split: test
2232
- revision: None
2233
- metrics:
2234
- - type: map_at_1
2235
- value: 0.198
2236
- - type: map_at_10
2237
- value: 1.3010000000000002
2238
- - type: map_at_100
2239
- value: 7.2139999999999995
2240
- - type: map_at_1000
2241
- value: 20.179
2242
- - type: map_at_3
2243
- value: 0.528
2244
- - type: map_at_5
2245
- value: 0.8019999999999999
2246
- - type: mrr_at_1
2247
- value: 72
2248
- - type: mrr_at_10
2249
- value: 83.39999999999999
2250
- - type: mrr_at_100
2251
- value: 83.39999999999999
2252
- - type: mrr_at_1000
2253
- value: 83.39999999999999
2254
- - type: mrr_at_3
2255
- value: 81.667
2256
- - type: mrr_at_5
2257
- value: 83.06700000000001
2258
- - type: ndcg_at_1
2259
- value: 66
2260
- - type: ndcg_at_10
2261
- value: 58.059000000000005
2262
- - type: ndcg_at_100
2263
- value: 44.316
2264
- - type: ndcg_at_1000
2265
- value: 43.147000000000006
2266
- - type: ndcg_at_3
2267
- value: 63.815999999999995
2268
- - type: ndcg_at_5
2269
- value: 63.005
2270
- - type: precision_at_1
2271
- value: 72
2272
- - type: precision_at_10
2273
- value: 61.4
2274
- - type: precision_at_100
2275
- value: 45.62
2276
- - type: precision_at_1000
2277
- value: 19.866
2278
- - type: precision_at_3
2279
- value: 70
2280
- - type: precision_at_5
2281
- value: 68.8
2282
- - type: recall_at_1
2283
- value: 0.198
2284
- - type: recall_at_10
2285
- value: 1.517
2286
- - type: recall_at_100
2287
- value: 10.587
2288
- - type: recall_at_1000
2289
- value: 41.233
2290
- - type: recall_at_3
2291
- value: 0.573
2292
- - type: recall_at_5
2293
- value: 0.907
2294
- - task:
2295
- type: Retrieval
2296
- dataset:
2297
- type: webis-touche2020
2298
- name: MTEB Touche2020
2299
- config: default
2300
- split: test
2301
- revision: None
2302
- metrics:
2303
- - type: map_at_1
2304
- value: 1.894
2305
- - type: map_at_10
2306
- value: 8.488999999999999
2307
- - type: map_at_100
2308
- value: 14.445
2309
- - type: map_at_1000
2310
- value: 16.078
2311
- - type: map_at_3
2312
- value: 4.589
2313
- - type: map_at_5
2314
- value: 6.019
2315
- - type: mrr_at_1
2316
- value: 22.448999999999998
2317
- - type: mrr_at_10
2318
- value: 39.82
2319
- - type: mrr_at_100
2320
- value: 40.752
2321
- - type: mrr_at_1000
2322
- value: 40.771
2323
- - type: mrr_at_3
2324
- value: 34.354
2325
- - type: mrr_at_5
2326
- value: 37.721
2327
- - type: ndcg_at_1
2328
- value: 19.387999999999998
2329
- - type: ndcg_at_10
2330
- value: 21.563
2331
- - type: ndcg_at_100
2332
- value: 33.857
2333
- - type: ndcg_at_1000
2334
- value: 46.199
2335
- - type: ndcg_at_3
2336
- value: 22.296
2337
- - type: ndcg_at_5
2338
- value: 21.770999999999997
2339
- - type: precision_at_1
2340
- value: 22.448999999999998
2341
- - type: precision_at_10
2342
- value: 19.796
2343
- - type: precision_at_100
2344
- value: 7.142999999999999
2345
- - type: precision_at_1000
2346
- value: 1.541
2347
- - type: precision_at_3
2348
- value: 24.490000000000002
2349
- - type: precision_at_5
2350
- value: 22.448999999999998
2351
- - type: recall_at_1
2352
- value: 1.894
2353
- - type: recall_at_10
2354
- value: 14.931
2355
- - type: recall_at_100
2356
- value: 45.524
2357
- - type: recall_at_1000
2358
- value: 83.243
2359
- - type: recall_at_3
2360
- value: 5.712
2361
- - type: recall_at_5
2362
- value: 8.386000000000001
2363
- - task:
2364
- type: Classification
2365
- dataset:
2366
- type: mteb/toxic_conversations_50k
2367
- name: MTEB ToxicConversationsClassification
2368
- config: default
2369
- split: test
2370
- revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
2371
- metrics:
2372
- - type: accuracy
2373
- value: 71.049
2374
- - type: ap
2375
- value: 13.85116971310922
2376
- - type: f1
2377
- value: 54.37504302487686
2378
- - task:
2379
- type: Classification
2380
- dataset:
2381
- type: mteb/tweet_sentiment_extraction
2382
- name: MTEB TweetSentimentExtractionClassification
2383
- config: default
2384
- split: test
2385
- revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
2386
- metrics:
2387
- - type: accuracy
2388
- value: 64.1312959818902
2389
- - type: f1
2390
- value: 64.11413877009383
2391
- - task:
2392
- type: Clustering
2393
- dataset:
2394
- type: mteb/twentynewsgroups-clustering
2395
- name: MTEB TwentyNewsgroupsClustering
2396
- config: default
2397
- split: test
2398
- revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
2399
- metrics:
2400
- - type: v_measure
2401
- value: 54.13103431861502
2402
- - task:
2403
- type: PairClassification
2404
- dataset:
2405
- type: mteb/twittersemeval2015-pairclassification
2406
- name: MTEB TwitterSemEval2015
2407
- config: default
2408
- split: test
2409
- revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
2410
- metrics:
2411
- - type: cos_sim_accuracy
2412
- value: 87.327889372355
2413
- - type: cos_sim_ap
2414
- value: 77.42059895975699
2415
- - type: cos_sim_f1
2416
- value: 71.02706903250873
2417
- - type: cos_sim_precision
2418
- value: 69.75324344950394
2419
- - type: cos_sim_recall
2420
- value: 72.34828496042216
2421
- - type: dot_accuracy
2422
- value: 87.327889372355
2423
- - type: dot_ap
2424
- value: 77.4209479346677
2425
- - type: dot_f1
2426
- value: 71.02706903250873
2427
- - type: dot_precision
2428
- value: 69.75324344950394
2429
- - type: dot_recall
2430
- value: 72.34828496042216
2431
- - type: euclidean_accuracy
2432
- value: 87.327889372355
2433
- - type: euclidean_ap
2434
- value: 77.42096495861037
2435
- - type: euclidean_f1
2436
- value: 71.02706903250873
2437
- - type: euclidean_precision
2438
- value: 69.75324344950394
2439
- - type: euclidean_recall
2440
- value: 72.34828496042216
2441
- - type: manhattan_accuracy
2442
- value: 87.31000774870358
2443
- - type: manhattan_ap
2444
- value: 77.38930750711619
2445
- - type: manhattan_f1
2446
- value: 71.07935314027831
2447
- - type: manhattan_precision
2448
- value: 67.70957726295677
2449
- - type: manhattan_recall
2450
- value: 74.80211081794195
2451
- - type: max_accuracy
2452
- value: 87.327889372355
2453
- - type: max_ap
2454
- value: 77.42096495861037
2455
- - type: max_f1
2456
- value: 71.07935314027831
2457
- - task:
2458
- type: PairClassification
2459
- dataset:
2460
- type: mteb/twitterurlcorpus-pairclassification
2461
- name: MTEB TwitterURLCorpus
2462
- config: default
2463
- split: test
2464
- revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
2465
- metrics:
2466
- - type: cos_sim_accuracy
2467
- value: 89.58939729110878
2468
- - type: cos_sim_ap
2469
- value: 87.17594155025475
2470
- - type: cos_sim_f1
2471
- value: 79.21146953405018
2472
- - type: cos_sim_precision
2473
- value: 76.8918527109307
2474
- - type: cos_sim_recall
2475
- value: 81.67539267015707
2476
- - type: dot_accuracy
2477
- value: 89.58939729110878
2478
- - type: dot_ap
2479
- value: 87.17593963273593
2480
- - type: dot_f1
2481
- value: 79.21146953405018
2482
- - type: dot_precision
2483
- value: 76.8918527109307
2484
- - type: dot_recall
2485
- value: 81.67539267015707
2486
- - type: euclidean_accuracy
2487
- value: 89.58939729110878
2488
- - type: euclidean_ap
2489
- value: 87.17592466925834
2490
- - type: euclidean_f1
2491
- value: 79.21146953405018
2492
- - type: euclidean_precision
2493
- value: 76.8918527109307
2494
- - type: euclidean_recall
2495
- value: 81.67539267015707
2496
- - type: manhattan_accuracy
2497
- value: 89.62626615438352
2498
- - type: manhattan_ap
2499
- value: 87.16589873161546
2500
- - type: manhattan_f1
2501
- value: 79.25143598295348
2502
- - type: manhattan_precision
2503
- value: 76.39494177323712
2504
- - type: manhattan_recall
2505
- value: 82.32984293193716
2506
- - type: max_accuracy
2507
- value: 89.62626615438352
2508
- - type: max_ap
2509
- value: 87.17594155025475
2510
- - type: max_f1
2511
- value: 79.25143598295348
2512
  ---
2513
 
2514
- # hkunlp/instructor-large
2515
- We introduce **Instructor**👨‍🏫, an instruction-finetuned text embedding model that can generate text embeddings tailored to any task (e.g., classification, retrieval, clustering, text evaluation, etc.) and domains (e.g., science, finance, etc.) ***by simply providing the task instruction, without any finetuning***. Instructor👨‍ achieves sota on 70 diverse embedding tasks ([MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard))!
2516
- The model is easy to use with **our customized** `sentence-transformer` library. For more details, check out [our paper](https://arxiv.org/abs/2212.09741) and [project page](https://instructor-embedding.github.io/)!
2517
 
2518
- **************************** **Updates** ****************************
 
 
2519
 
2520
- * 12/28: We released a new [checkpoint](https://huggingface.co/hkunlp/instructor-large) trained with hard negatives, which gives better performance.
2521
- * 12/21: We released our [paper](https://arxiv.org/abs/2212.09741), [code](https://github.com/HKUNLP/instructor-embedding), [checkpoint](https://huggingface.co/hkunlp/instructor-large) and [project page](https://instructor-embedding.github.io/)! Check them out!
2522
 
2523
- ## Quick start
2524
- <hr />
2525
 
2526
- ## Installation
2527
- ```bash
2528
- pip install InstructorEmbedding
2529
- ```
2530
 
2531
- ## Compute your customized embeddings
2532
- Then you can use the model like this to calculate domain-specific and task-aware embeddings:
2533
- ```python
2534
- from InstructorEmbedding import INSTRUCTOR
2535
- model = INSTRUCTOR('hkunlp/instructor-large')
2536
- sentence = "3D ActionSLAM: wearable person tracking in multi-floor environments"
2537
- instruction = "Represent the Science title:"
2538
- embeddings = model.encode([[instruction,sentence]])
2539
- print(embeddings)
2540
- ```
2541
 
2542
- ## Use cases
2543
- <hr />
 
 
 
2544
 
2545
- ## Calculate embeddings for your customized texts
2546
- If you want to calculate customized embeddings for specific sentences, you may follow the unified template to write instructions:
2547
 
2548
- &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Represent the `domain` `text_type` for `task_objective`:
2549
- * `domain` is optional, and it specifies the domain of the text, e.g., science, finance, medicine, etc.
2550
- * `text_type` is required, and it specifies the encoding unit, e.g., sentence, document, paragraph, etc.
2551
- * `task_objective` is optional, and it specifies the objective of embedding, e.g., retrieve a document, classify the sentence, etc.
 
2552
 
2553
- ## Calculate Sentence similarities
2554
- You can further use the model to compute similarities between two groups of sentences, with **customized embeddings**.
2555
- ```python
2556
- from sklearn.metrics.pairwise import cosine_similarity
2557
- sentences_a = [['Represent the Science sentence: ','Parton energy loss in QCD matter'],
2558
- ['Represent the Financial statement: ','The Federal Reserve on Wednesday raised its benchmark interest rate.']]
2559
- sentences_b = [['Represent the Science sentence: ','The Chiral Phase Transition in Dissipative Dynamics'],
2560
- ['Represent the Financial statement: ','The funds rose less than 0.5 per cent on Friday']]
2561
- embeddings_a = model.encode(sentences_a)
2562
- embeddings_b = model.encode(sentences_b)
2563
- similarities = cosine_similarity(embeddings_a,embeddings_b)
2564
- print(similarities)
2565
- ```
2566
 
2567
- ## Information Retrieval
2568
- You can also use **customized embeddings** for information retrieval.
2569
  ```python
2570
- import numpy as np
2571
- from sklearn.metrics.pairwise import cosine_similarity
2572
- query = [['Represent the Wikipedia question for retrieving supporting documents: ','where is the food stored in a yam plant']]
2573
- corpus = [['Represent the Wikipedia document for retrieval: ','Capitalism has been dominant in the Western world since the end of feudalism, but most feel[who?] that the term "mixed economies" more precisely describes most contemporary economies, due to their containing both private-owned and state-owned enterprises. In capitalism, prices determine the demand-supply scale. For example, higher demand for certain goods and services lead to higher prices and lower demand for certain goods lead to lower prices.'],
2574
- ['Represent the Wikipedia document for retrieval: ',"The disparate impact theory is especially controversial under the Fair Housing Act because the Act regulates many activities relating to housing, insurance, and mortgage loans—and some scholars have argued that the theory's use under the Fair Housing Act, combined with extensions of the Community Reinvestment Act, contributed to rise of sub-prime lending and the crash of the U.S. housing market and ensuing global economic recession"],
2575
- ['Represent the Wikipedia document for retrieval: ','Disparate impact in United States labor law refers to practices in employment, housing, and other areas that adversely affect one group of people of a protected characteristic more than another, even though rules applied by employers or landlords are formally neutral. Although the protected classes vary by statute, most federal civil rights laws protect based on race, color, religion, national origin, and sex as protected traits, and some laws include disability status and other traits as well.']]
2576
- query_embeddings = model.encode(query)
2577
- corpus_embeddings = model.encode(corpus)
2578
- similarities = cosine_similarity(query_embeddings,corpus_embeddings)
2579
- retrieved_doc_id = np.argmax(similarities)
2580
- print(retrieved_doc_id)
 
 
 
2581
  ```
2582
 
2583
- ## Clustering
2584
- Use **customized embeddings** for clustering texts in groups.
2585
- ```python
2586
- import sklearn.cluster
2587
- sentences = [['Represent the Medicine sentence for clustering: ','Dynamical Scalar Degree of Freedom in Horava-Lifshitz Gravity'],
2588
- ['Represent the Medicine sentence for clustering: ','Comparison of Atmospheric Neutrino Flux Calculations at Low Energies'],
2589
- ['Represent the Medicine sentence for clustering: ','Fermion Bags in the Massive Gross-Neveu Model'],
2590
- ['Represent the Medicine sentence for clustering: ',"QCD corrections to Associated t-tbar-H production at the Tevatron"],
2591
- ['Represent the Medicine sentence for clustering: ','A New Analysis of the R Measurements: Resonance Parameters of the Higher, Vector States of Charmonium']]
2592
- embeddings = model.encode(sentences)
2593
- clustering_model = sklearn.cluster.MiniBatchKMeans(n_clusters=2)
2594
- clustering_model.fit(embeddings)
2595
- cluster_assignment = clustering_model.labels_
2596
- print(cluster_assignment)
2597
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  - dementia
13
  - dementia disease
14
  language: en
15
+ inference: true
16
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
  ---
18
 
19
+ # **My LLM Model: Dementia Knowledge Assistant**
 
 
20
 
21
+ **Model Name:** `Dementia-llm-model`
22
+ **Description:**
23
+ This is a fine-tuned **Large Language Model (LLM)** designed to assist with dementia-related knowledge retrieval and question-answering tasks. The model uses advanced embeddings (`hkunlp/instructor-large`) and a **FAISS vector store** for efficient contextual search and retrieval.
24
 
25
+ ---
 
26
 
27
+ ## **Model Summary**
 
28
 
29
+ This LLM is fine-tuned on a dataset specifically curated for dementia-related content, including medical knowledge, patient care, and treatment practices. It leverages state-of-the-art embeddings to generate accurate and contextually relevant answers to user queries. The model supports researchers, caregivers, and medical professionals in accessing domain-specific information quickly.
 
 
 
30
 
31
+ ---
 
 
 
 
 
 
 
 
 
32
 
33
+ ## **Key Features**
34
+ - **Domain-Specific Knowledge:** Trained on a dementia-related dataset for precise answers.
35
+ - **Embeddings:** Utilizes the `hkunlp/instructor-large` embedding model for semantic understanding.
36
+ - **Retrieval-augmented QA:** Employs FAISS vector databases for efficient document retrieval.
37
+ - **Custom Prompting:** Generates responses based on well-designed prompts to ensure factual accuracy.
38
 
39
+ ---
 
40
 
41
+ ## **Intended Use**
42
+ - **Primary Use Case:** Question-answering related to dementia.
43
+ - **Secondary Use Cases:** Exploring dementia knowledge, aiding medical students or caregivers in understanding dementia-related topics, and supporting researchers.
44
+ - **Input Format:** Text queries in natural language.
45
+ - **Output Format:** Natural language responses relevant to the context provided.
46
 
47
+ ---
48
+
49
+ ## **Limitations**
50
+ - **Context Dependency:** Model outputs are only as good as the context provided by the FAISS retriever. If the context is insufficient, the model may respond with "I don't know."
51
+ - **Static Knowledge:** The model is limited to the knowledge present in its training dataset. It may not include the latest medical breakthroughs or research after the training cutoff.
52
+ - **Biases:** The model might inherit biases present in the training data.
53
+
54
+ ---
55
+
56
+ ## **How to Use**
57
+
58
+ ### **Using the Model Programmatically**
59
+ You can use the model directly in Python:
60
 
 
 
61
  ```python
62
+ from transformers import pipeline
63
+
64
+ model_name = "rohitashva/my-llm-model"
65
+
66
+ # Load the model and tokenizer
67
+ qa_pipeline = pipeline("question-answering", model=model_name)
68
+
69
+ # Example Query
70
+ result = qa_pipeline({
71
+ "question": "What are the symptoms of early-stage dementia?",
72
+ "context": "Provide relevant details from a dementia dataset."
73
+ })
74
+
75
+ print(result)
76
  ```
77
 
78
+ ---
79
+ ### **Training Details**
80
+
81
+ • Base Model: hkunlp/instructor-large
82
+ • Frameworks: PyTorch, Transformers
83
+ • Embedding Model: HuggingFace Embeddings (hkunlp/instructor-large)
84
+ • Fine-Tuning: FAISS-based vector retrieval augmented with dementia-specific content.
85
+ • Hardware: Trained on a GPU with sufficient VRAM for embeddings and fine-tuning tasks.
86
+
87
+ ---
88
+
89
+
90
+ ## Further Information
91
+
92
+ ### Dataset
93
+
94
+ The model was trained on a proprietary dementia-specific dataset, including structured knowledge, medical texts, and patient case studies. The data is preprocessed into embeddings for efficient retrieval.
95
+
96
+ ### Model Performance
97
+
98
+ • Accuracy: Validated on a subset of dementia-related QA pairs.
99
+ • Response Time: Optimized for fast retrieval via FAISS vector storage.
100
+
101
+ ### Deployment
102
+
103
+ • Hugging Face Spaces: The model is deployed on Hugging Face Spaces, enabling users to interact via a web-based interface.
104
+ • API Support: The model is available for integration into custom workflows using the Hugging Face Inference API.
105
+
106
+ ### Acknowledgments
107
+
108
+ • Hugging Face team for the transformers library.
109
+ • Contributors to the hkunlp/instructor-large embedding model.
110
+ • Medical experts and datasets used for model fine-tuning.
111
+