Upload README.md
Browse files
README.md
CHANGED
@@ -1,4 +1,5 @@
|
|
1 |
---
|
|
|
2 |
model-index:
|
3 |
- name: XYZ-embedding-zh
|
4 |
results:
|
@@ -10,11 +11,11 @@ model-index:
|
|
10 |
type: C-MTEB/CMedQAv1-reranking
|
11 |
metrics:
|
12 |
- type: map
|
13 |
-
value: 89.
|
14 |
- type: mrr
|
15 |
-
value: 91.
|
16 |
- type: main_score
|
17 |
-
value: 89.
|
18 |
task:
|
19 |
type: Reranking
|
20 |
- dataset:
|
@@ -25,11 +26,11 @@ model-index:
|
|
25 |
type: C-MTEB/CMedQAv2-reranking
|
26 |
metrics:
|
27 |
- type: map
|
28 |
-
value: 89.
|
29 |
- type: mrr
|
30 |
-
value: 91.
|
31 |
- type: main_score
|
32 |
-
value: 89.
|
33 |
task:
|
34 |
type: Reranking
|
35 |
- dataset:
|
@@ -40,67 +41,67 @@ model-index:
|
|
40 |
type: C-MTEB/CmedqaRetrieval
|
41 |
metrics:
|
42 |
- type: map_at_1
|
43 |
-
value: 27.
|
44 |
- type: map_at_10
|
45 |
-
value: 41.
|
46 |
- type: map_at_100
|
47 |
-
value: 43.
|
48 |
- type: map_at_1000
|
49 |
-
value: 43.
|
50 |
- type: map_at_3
|
51 |
-
value:
|
52 |
- type: map_at_5
|
53 |
-
value: 39.
|
54 |
- type: mrr_at_1
|
55 |
-
value: 42.
|
56 |
- type: mrr_at_10
|
57 |
-
value: 50.
|
58 |
- type: mrr_at_100
|
59 |
-
value: 51.
|
60 |
- type: mrr_at_1000
|
61 |
-
value: 51.
|
62 |
- type: mrr_at_3
|
63 |
-
value:
|
64 |
- type: mrr_at_5
|
65 |
-
value: 49.
|
66 |
- type: ndcg_at_1
|
67 |
-
value: 42.
|
68 |
- type: ndcg_at_10
|
69 |
-
value:
|
70 |
- type: ndcg_at_100
|
71 |
-
value:
|
72 |
- type: ndcg_at_1000
|
73 |
-
value: 56.
|
74 |
- type: ndcg_at_3
|
75 |
-
value:
|
76 |
- type: ndcg_at_5
|
77 |
-
value:
|
78 |
- type: precision_at_1
|
79 |
-
value: 42.
|
80 |
- type: precision_at_10
|
81 |
-
value: 10.
|
82 |
- type: precision_at_100
|
83 |
-
value: 1.
|
84 |
- type: precision_at_1000
|
85 |
value: 0.183
|
86 |
- type: precision_at_3
|
87 |
-
value: 24.
|
88 |
- type: precision_at_5
|
89 |
-
value: 17.
|
90 |
- type: recall_at_1
|
91 |
-
value: 27.
|
92 |
- type: recall_at_10
|
93 |
-
value:
|
94 |
- type: recall_at_100
|
95 |
-
value: 86.
|
96 |
- type: recall_at_1000
|
97 |
-
value: 98.
|
98 |
- type: recall_at_3
|
99 |
-
value:
|
100 |
- type: recall_at_5
|
101 |
-
value: 49.
|
102 |
- type: main_score
|
103 |
-
value:
|
104 |
task:
|
105 |
type: Retrieval
|
106 |
- dataset:
|
@@ -111,67 +112,67 @@ model-index:
|
|
111 |
type: C-MTEB/CovidRetrieval
|
112 |
metrics:
|
113 |
- type: map_at_1
|
114 |
-
value:
|
115 |
- type: map_at_10
|
116 |
-
value:
|
117 |
- type: map_at_100
|
118 |
-
value:
|
119 |
- type: map_at_1000
|
120 |
-
value:
|
121 |
- type: map_at_3
|
122 |
-
value:
|
123 |
- type: map_at_5
|
124 |
-
value:
|
125 |
- type: mrr_at_1
|
126 |
-
value:
|
127 |
- type: mrr_at_10
|
128 |
-
value:
|
129 |
- type: mrr_at_100
|
130 |
-
value:
|
131 |
- type: mrr_at_1000
|
132 |
-
value:
|
133 |
- type: mrr_at_3
|
134 |
-
value:
|
135 |
- type: mrr_at_5
|
136 |
-
value:
|
137 |
- type: ndcg_at_1
|
138 |
-
value:
|
139 |
- type: ndcg_at_10
|
140 |
-
value:
|
141 |
- type: ndcg_at_100
|
142 |
-
value:
|
143 |
- type: ndcg_at_1000
|
144 |
-
value:
|
145 |
- type: ndcg_at_3
|
146 |
-
value:
|
147 |
- type: ndcg_at_5
|
148 |
-
value:
|
149 |
- type: precision_at_1
|
150 |
-
value:
|
151 |
- type: precision_at_10
|
152 |
-
value:
|
153 |
- type: precision_at_100
|
154 |
-
value: 1.
|
155 |
- type: precision_at_1000
|
156 |
value: 0.101
|
157 |
- type: precision_at_3
|
158 |
-
value:
|
159 |
- type: precision_at_5
|
160 |
-
value: 19.
|
161 |
- type: recall_at_1
|
162 |
-
value:
|
163 |
- type: recall_at_10
|
164 |
-
value:
|
165 |
- type: recall_at_100
|
166 |
-
value: 99.
|
167 |
- type: recall_at_1000
|
168 |
value: 100.0
|
169 |
- type: recall_at_3
|
170 |
-
value:
|
171 |
- type: recall_at_5
|
172 |
-
value:
|
173 |
- type: main_score
|
174 |
-
value:
|
175 |
task:
|
176 |
type: Retrieval
|
177 |
- dataset:
|
@@ -182,67 +183,67 @@ model-index:
|
|
182 |
type: C-MTEB/DuRetrieval
|
183 |
metrics:
|
184 |
- type: map_at_1
|
185 |
-
value: 27.
|
186 |
- type: map_at_10
|
187 |
-
value:
|
188 |
- type: map_at_100
|
189 |
-
value: 87.
|
190 |
- type: map_at_1000
|
191 |
-
value: 87.
|
192 |
- type: map_at_3
|
193 |
-
value: 59.
|
194 |
- type: map_at_5
|
195 |
-
value:
|
196 |
- type: mrr_at_1
|
197 |
value: 93.65
|
198 |
- type: mrr_at_10
|
199 |
-
value: 95.
|
200 |
- type: mrr_at_100
|
201 |
-
value: 95.
|
202 |
- type: mrr_at_1000
|
203 |
-
value: 95.
|
204 |
- type: mrr_at_3
|
205 |
-
value: 95.
|
206 |
- type: mrr_at_5
|
207 |
-
value: 95.
|
208 |
- type: ndcg_at_1
|
209 |
value: 93.65
|
210 |
- type: ndcg_at_10
|
211 |
-
value:
|
212 |
- type: ndcg_at_100
|
213 |
-
value:
|
214 |
- type: ndcg_at_1000
|
215 |
-
value: 93.
|
216 |
- type: ndcg_at_3
|
217 |
-
value: 90.
|
218 |
- type: ndcg_at_5
|
219 |
-
value: 89.
|
220 |
- type: precision_at_1
|
221 |
value: 93.65
|
222 |
- type: precision_at_10
|
223 |
-
value:
|
224 |
- type: precision_at_100
|
225 |
-
value: 4.
|
226 |
- type: precision_at_1000
|
227 |
-
value: 0.
|
228 |
- type: precision_at_3
|
229 |
-
value:
|
230 |
- type: precision_at_5
|
231 |
-
value: 68.
|
232 |
- type: recall_at_1
|
233 |
-
value: 27.
|
234 |
- type: recall_at_10
|
235 |
-
value: 91.
|
236 |
- type: recall_at_100
|
237 |
-
value: 98.
|
238 |
- type: recall_at_1000
|
239 |
-
value: 99.
|
240 |
- type: recall_at_3
|
241 |
-
value:
|
242 |
- type: recall_at_5
|
243 |
-
value: 78.
|
244 |
- type: main_score
|
245 |
-
value:
|
246 |
task:
|
247 |
type: Retrieval
|
248 |
- dataset:
|
@@ -253,67 +254,67 @@ model-index:
|
|
253 |
type: C-MTEB/EcomRetrieval
|
254 |
metrics:
|
255 |
- type: map_at_1
|
256 |
-
value: 54.
|
257 |
- type: map_at_10
|
258 |
-
value:
|
259 |
- type: map_at_100
|
260 |
-
value: 65.
|
261 |
- type: map_at_1000
|
262 |
-
value: 65.
|
263 |
- type: map_at_3
|
264 |
-
value: 62.
|
265 |
- type: map_at_5
|
266 |
-
value:
|
267 |
- type: mrr_at_1
|
268 |
-
value: 54.
|
269 |
- type: mrr_at_10
|
270 |
-
value:
|
271 |
- type: mrr_at_100
|
272 |
-
value: 65.
|
273 |
- type: mrr_at_1000
|
274 |
-
value: 65.
|
275 |
- type: mrr_at_3
|
276 |
-
value: 62.
|
277 |
- type: mrr_at_5
|
278 |
-
value:
|
279 |
- type: ndcg_at_1
|
280 |
-
value: 54.
|
281 |
- type: ndcg_at_10
|
282 |
-
value:
|
283 |
- type: ndcg_at_100
|
284 |
-
value: 72.
|
285 |
- type: ndcg_at_1000
|
286 |
-
value: 72.
|
287 |
- type: ndcg_at_3
|
288 |
-
value:
|
289 |
- type: ndcg_at_5
|
290 |
-
value: 67.
|
291 |
- type: precision_at_1
|
292 |
-
value: 54.
|
293 |
- type: precision_at_10
|
294 |
-
value: 8.
|
295 |
- type: precision_at_100
|
296 |
-
value: 0.
|
297 |
- type: precision_at_1000
|
298 |
-
value: 0.
|
299 |
- type: precision_at_3
|
300 |
-
value:
|
301 |
- type: precision_at_5
|
302 |
-
value: 15.
|
303 |
- type: recall_at_1
|
304 |
-
value: 54.
|
305 |
- type: recall_at_10
|
306 |
-
value:
|
307 |
- type: recall_at_100
|
308 |
-
value: 96.
|
309 |
- type: recall_at_1000
|
310 |
-
value: 98.
|
311 |
- type: recall_at_3
|
312 |
-
value:
|
313 |
- type: recall_at_5
|
314 |
-
value: 79.
|
315 |
- type: main_score
|
316 |
-
value:
|
317 |
task:
|
318 |
type: Retrieval
|
319 |
- dataset:
|
@@ -324,11 +325,11 @@ model-index:
|
|
324 |
type: C-MTEB/Mmarco-reranking
|
325 |
metrics:
|
326 |
- type: map
|
327 |
-
value:
|
328 |
- type: mrr
|
329 |
-
value:
|
330 |
- type: main_score
|
331 |
-
value:
|
332 |
task:
|
333 |
type: Reranking
|
334 |
- dataset:
|
@@ -339,67 +340,67 @@ model-index:
|
|
339 |
type: C-MTEB/MMarcoRetrieval
|
340 |
metrics:
|
341 |
- type: map_at_1
|
342 |
-
value: 69.
|
343 |
- type: map_at_10
|
344 |
-
value: 78.
|
345 |
- type: map_at_100
|
346 |
-
value: 79.
|
347 |
- type: map_at_1000
|
348 |
-
value: 79.
|
349 |
- type: map_at_3
|
350 |
-
value: 76.
|
351 |
- type: map_at_5
|
352 |
-
value: 78.
|
353 |
- type: mrr_at_1
|
354 |
-
value: 71.
|
355 |
- type: mrr_at_10
|
356 |
-
value: 79.
|
357 |
- type: mrr_at_100
|
358 |
-
value: 79.
|
359 |
- type: mrr_at_1000
|
360 |
-
value: 79.
|
361 |
- type: mrr_at_3
|
362 |
-
value: 77.
|
363 |
- type: mrr_at_5
|
364 |
-
value: 78.
|
365 |
- type: ndcg_at_1
|
366 |
-
value: 71.
|
367 |
- type: ndcg_at_10
|
368 |
-
value: 82.
|
369 |
- type: ndcg_at_100
|
370 |
-
value: 83.
|
371 |
- type: ndcg_at_1000
|
372 |
-
value: 83.
|
373 |
- type: ndcg_at_3
|
374 |
-
value: 79.
|
375 |
- type: ndcg_at_5
|
376 |
-
value: 81.
|
377 |
- type: precision_at_1
|
378 |
-
value: 71.
|
379 |
- type: precision_at_10
|
380 |
-
value: 9.
|
381 |
- type: precision_at_100
|
382 |
-
value: 1.
|
383 |
- type: precision_at_1000
|
384 |
value: 0.106
|
385 |
- type: precision_at_3
|
386 |
-
value: 29.
|
387 |
- type: precision_at_5
|
388 |
-
value: 18.
|
389 |
- type: recall_at_1
|
390 |
-
value: 69.
|
391 |
- type: recall_at_10
|
392 |
-
value: 93.
|
393 |
- type: recall_at_100
|
394 |
-
value: 98.
|
395 |
- type: recall_at_1000
|
396 |
-
value: 99.
|
397 |
- type: recall_at_3
|
398 |
-
value: 84.
|
399 |
- type: recall_at_5
|
400 |
-
value: 89.
|
401 |
- type: main_score
|
402 |
-
value: 82.
|
403 |
task:
|
404 |
type: Retrieval
|
405 |
- dataset:
|
@@ -412,65 +413,65 @@ model-index:
|
|
412 |
- type: map_at_1
|
413 |
value: 57.8
|
414 |
- type: map_at_10
|
415 |
-
value: 64.
|
416 |
- type: map_at_100
|
417 |
-
value:
|
418 |
- type: map_at_1000
|
419 |
-
value:
|
420 |
- type: map_at_3
|
421 |
-
value: 62.
|
422 |
- type: map_at_5
|
423 |
-
value:
|
424 |
- type: mrr_at_1
|
425 |
value: 58.099999999999994
|
426 |
- type: mrr_at_10
|
427 |
-
value: 64.
|
428 |
- type: mrr_at_100
|
429 |
-
value:
|
430 |
- type: mrr_at_1000
|
431 |
-
value:
|
432 |
- type: mrr_at_3
|
433 |
-
value:
|
434 |
- type: mrr_at_5
|
435 |
-
value:
|
436 |
- type: ndcg_at_1
|
437 |
value: 57.8
|
438 |
- type: ndcg_at_10
|
439 |
-
value:
|
440 |
- type: ndcg_at_100
|
441 |
-
value:
|
442 |
- type: ndcg_at_1000
|
443 |
-
value: 71.
|
444 |
- type: ndcg_at_3
|
445 |
-
value: 64.
|
446 |
- type: ndcg_at_5
|
447 |
-
value:
|
448 |
- type: precision_at_1
|
449 |
value: 57.8
|
450 |
- type: precision_at_10
|
451 |
-
value: 7.
|
452 |
- type: precision_at_100
|
453 |
-
value: 0.
|
454 |
- type: precision_at_1000
|
455 |
value: 0.099
|
456 |
- type: precision_at_3
|
457 |
-
value:
|
458 |
- type: precision_at_5
|
459 |
-
value: 14.
|
460 |
- type: recall_at_1
|
461 |
value: 57.8
|
462 |
- type: recall_at_10
|
463 |
-
value:
|
464 |
- type: recall_at_100
|
465 |
-
value:
|
466 |
- type: recall_at_1000
|
467 |
-
value:
|
468 |
- type: recall_at_3
|
469 |
-
value:
|
470 |
- type: recall_at_5
|
471 |
-
value:
|
472 |
- type: main_score
|
473 |
-
value:
|
474 |
task:
|
475 |
type: Retrieval
|
476 |
- dataset:
|
@@ -481,11 +482,11 @@ model-index:
|
|
481 |
type: C-MTEB/T2Reranking
|
482 |
metrics:
|
483 |
- type: map
|
484 |
-
value: 69.
|
485 |
- type: mrr
|
486 |
-
value: 79.
|
487 |
- type: main_score
|
488 |
-
value: 69.
|
489 |
task:
|
490 |
type: Reranking
|
491 |
- dataset:
|
@@ -496,67 +497,67 @@ model-index:
|
|
496 |
type: C-MTEB/T2Retrieval
|
497 |
metrics:
|
498 |
- type: map_at_1
|
499 |
-
value: 28.
|
500 |
- type: map_at_10
|
501 |
-
value: 78.
|
502 |
- type: map_at_100
|
503 |
-
value: 82.
|
504 |
- type: map_at_1000
|
505 |
-
value: 82.
|
506 |
- type: map_at_3
|
507 |
-
value: 55.
|
508 |
- type: map_at_5
|
509 |
-
value: 67.
|
510 |
- type: mrr_at_1
|
511 |
-
value:
|
512 |
- type: mrr_at_10
|
513 |
-
value: 93.
|
514 |
- type: mrr_at_100
|
515 |
-
value: 93.
|
516 |
- type: mrr_at_1000
|
517 |
-
value: 93.
|
518 |
- type: mrr_at_3
|
519 |
-
value: 92.
|
520 |
- type: mrr_at_5
|
521 |
-
value: 93.
|
522 |
- type: ndcg_at_1
|
523 |
-
value:
|
524 |
- type: ndcg_at_10
|
525 |
-
value: 85.
|
526 |
- type: ndcg_at_100
|
527 |
-
value: 89.
|
528 |
- type: ndcg_at_1000
|
529 |
-
value: 89.
|
530 |
- type: ndcg_at_3
|
531 |
-
value: 87.
|
532 |
- type: ndcg_at_5
|
533 |
-
value: 85.
|
534 |
- type: precision_at_1
|
535 |
-
value:
|
536 |
- type: precision_at_10
|
537 |
-
value: 42.
|
538 |
- type: precision_at_100
|
539 |
-
value: 5.
|
540 |
- type: precision_at_1000
|
541 |
value: 0.516
|
542 |
- type: precision_at_3
|
543 |
-
value: 76.
|
544 |
- type: precision_at_5
|
545 |
-
value: 63.
|
546 |
- type: recall_at_1
|
547 |
-
value: 28.
|
548 |
- type: recall_at_10
|
549 |
-
value: 84.
|
550 |
- type: recall_at_100
|
551 |
-
value: 95.
|
552 |
- type: recall_at_1000
|
553 |
-
value: 98.
|
554 |
- type: recall_at_3
|
555 |
-
value: 57.
|
556 |
- type: recall_at_5
|
557 |
-
value: 71.
|
558 |
- type: main_score
|
559 |
-
value: 85.
|
560 |
task:
|
561 |
type: Retrieval
|
562 |
- dataset:
|
@@ -567,99 +568,121 @@ model-index:
|
|
567 |
type: C-MTEB/VideoRetrieval
|
568 |
metrics:
|
569 |
- type: map_at_1
|
570 |
-
value:
|
571 |
- type: map_at_10
|
572 |
-
value:
|
573 |
- type: map_at_100
|
574 |
-
value:
|
575 |
- type: map_at_1000
|
576 |
-
value:
|
577 |
- type: map_at_3
|
578 |
-
value:
|
579 |
- type: map_at_5
|
580 |
-
value:
|
581 |
- type: mrr_at_1
|
582 |
-
value:
|
583 |
- type: mrr_at_10
|
584 |
-
value:
|
585 |
- type: mrr_at_100
|
586 |
-
value:
|
587 |
- type: mrr_at_1000
|
588 |
-
value:
|
589 |
- type: mrr_at_3
|
590 |
-
value:
|
591 |
- type: mrr_at_5
|
592 |
-
value:
|
593 |
- type: ndcg_at_1
|
594 |
-
value:
|
595 |
- type: ndcg_at_10
|
596 |
-
value: 80.
|
597 |
- type: ndcg_at_100
|
598 |
-
value:
|
599 |
- type: ndcg_at_1000
|
600 |
-
value:
|
601 |
- type: ndcg_at_3
|
602 |
-
value:
|
603 |
- type: ndcg_at_5
|
604 |
-
value:
|
605 |
- type: precision_at_1
|
606 |
-
value:
|
607 |
- type: precision_at_10
|
608 |
-
value: 9.
|
609 |
- type: precision_at_100
|
610 |
-
value: 0.
|
611 |
- type: precision_at_1000
|
612 |
value: 0.1
|
613 |
- type: precision_at_3
|
614 |
-
value:
|
615 |
- type: precision_at_5
|
616 |
-
value: 17.
|
617 |
- type: recall_at_1
|
618 |
-
value:
|
619 |
- type: recall_at_10
|
620 |
-
value:
|
621 |
- type: recall_at_100
|
622 |
-
value: 99.
|
623 |
- type: recall_at_1000
|
624 |
-
value: 99.
|
625 |
- type: recall_at_3
|
626 |
-
value:
|
627 |
- type: recall_at_5
|
628 |
-
value:
|
629 |
- type: main_score
|
630 |
-
value: 80.
|
631 |
task:
|
632 |
type: Retrieval
|
|
|
633 |
tags:
|
|
|
|
|
|
|
634 |
- mteb
|
635 |
---
|
636 |
-
|
|
|
|
|
637 |
|
638 |
-
|
639 |
|
640 |
-
|
641 |
|
642 |
-
|
|
|
|
|
|
|
|
|
643 |
pip install -U sentence-transformers
|
644 |
```
|
645 |
-
|
|
|
|
|
646 |
```python
|
647 |
from sentence_transformers import SentenceTransformer
|
|
|
648 |
|
649 |
-
|
650 |
-
model = SentenceTransformer("fangxq/XYZ-embedding-zh")
|
651 |
-
# Run inference
|
652 |
-
sentences = [
|
653 |
-
'The weather is lovely today.',
|
654 |
-
"It's so sunny outside!",
|
655 |
-
'He drove to the stadium.',
|
656 |
-
]
|
657 |
embeddings = model.encode(sentences)
|
658 |
-
print(embeddings
|
659 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
660 |
|
661 |
-
|
662 |
-
|
663 |
-
|
664 |
-
|
|
|
|
|
|
|
|
|
|
|
665 |
```
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
library_name: sentence-transformers
|
3 |
model-index:
|
4 |
- name: XYZ-embedding-zh
|
5 |
results:
|
|
|
11 |
type: C-MTEB/CMedQAv1-reranking
|
12 |
metrics:
|
13 |
- type: map
|
14 |
+
value: 89.61792115239176
|
15 |
- type: mrr
|
16 |
+
value: 91.46722222222222
|
17 |
- type: main_score
|
18 |
+
value: 89.61792115239176
|
19 |
task:
|
20 |
type: Reranking
|
21 |
- dataset:
|
|
|
26 |
type: C-MTEB/CMedQAv2-reranking
|
27 |
metrics:
|
28 |
- type: map
|
29 |
+
value: 89.22040591564271
|
30 |
- type: mrr
|
31 |
+
value: 91.2995238095238
|
32 |
- type: main_score
|
33 |
+
value: 89.22040591564271
|
34 |
task:
|
35 |
type: Reranking
|
36 |
- dataset:
|
|
|
41 |
type: C-MTEB/CmedqaRetrieval
|
42 |
metrics:
|
43 |
- type: map_at_1
|
44 |
+
value: 27.939000000000004
|
45 |
- type: map_at_10
|
46 |
+
value: 41.227999999999994
|
47 |
- type: map_at_100
|
48 |
+
value: 43.018
|
49 |
- type: map_at_1000
|
50 |
+
value: 43.120000000000005
|
51 |
- type: map_at_3
|
52 |
+
value: 36.895
|
53 |
- type: map_at_5
|
54 |
+
value: 39.373999999999995
|
55 |
- type: mrr_at_1
|
56 |
+
value: 42.136
|
57 |
- type: mrr_at_10
|
58 |
+
value: 50.394000000000005
|
59 |
- type: mrr_at_100
|
60 |
+
value: 51.288
|
61 |
- type: mrr_at_1000
|
62 |
+
value: 51.324000000000005
|
63 |
- type: mrr_at_3
|
64 |
+
value: 47.887
|
65 |
- type: mrr_at_5
|
66 |
+
value: 49.362
|
67 |
- type: ndcg_at_1
|
68 |
+
value: 42.136
|
69 |
- type: ndcg_at_10
|
70 |
+
value: 47.899
|
71 |
- type: ndcg_at_100
|
72 |
+
value: 54.730999999999995
|
73 |
- type: ndcg_at_1000
|
74 |
+
value: 56.462999999999994
|
75 |
- type: ndcg_at_3
|
76 |
+
value: 42.66
|
77 |
- type: ndcg_at_5
|
78 |
+
value: 44.913
|
79 |
- type: precision_at_1
|
80 |
+
value: 42.136
|
81 |
- type: precision_at_10
|
82 |
+
value: 10.52
|
83 |
- type: precision_at_100
|
84 |
+
value: 1.6070000000000002
|
85 |
- type: precision_at_1000
|
86 |
value: 0.183
|
87 |
- type: precision_at_3
|
88 |
+
value: 24.064
|
89 |
- type: precision_at_5
|
90 |
+
value: 17.374000000000002
|
91 |
- type: recall_at_1
|
92 |
+
value: 27.939000000000004
|
93 |
- type: recall_at_10
|
94 |
+
value: 58.29600000000001
|
95 |
- type: recall_at_100
|
96 |
+
value: 86.504
|
97 |
- type: recall_at_1000
|
98 |
+
value: 98.105
|
99 |
- type: recall_at_3
|
100 |
+
value: 42.475
|
101 |
- type: recall_at_5
|
102 |
+
value: 49.454
|
103 |
- type: main_score
|
104 |
+
value: 47.899
|
105 |
task:
|
106 |
type: Retrieval
|
107 |
- dataset:
|
|
|
112 |
type: C-MTEB/CovidRetrieval
|
113 |
metrics:
|
114 |
- type: map_at_1
|
115 |
+
value: 77.371
|
116 |
- type: map_at_10
|
117 |
+
value: 85.229
|
118 |
- type: map_at_100
|
119 |
+
value: 85.358
|
120 |
- type: map_at_1000
|
121 |
+
value: 85.36
|
122 |
- type: map_at_3
|
123 |
+
value: 84.176
|
124 |
- type: map_at_5
|
125 |
+
value: 84.79299999999999
|
126 |
- type: mrr_at_1
|
127 |
+
value: 77.661
|
128 |
- type: mrr_at_10
|
129 |
+
value: 85.207
|
130 |
- type: mrr_at_100
|
131 |
+
value: 85.33699999999999
|
132 |
- type: mrr_at_1000
|
133 |
+
value: 85.339
|
134 |
- type: mrr_at_3
|
135 |
+
value: 84.229
|
136 |
- type: mrr_at_5
|
137 |
+
value: 84.79299999999999
|
138 |
- type: ndcg_at_1
|
139 |
+
value: 77.766
|
140 |
- type: ndcg_at_10
|
141 |
+
value: 88.237
|
142 |
- type: ndcg_at_100
|
143 |
+
value: 88.777
|
144 |
- type: ndcg_at_1000
|
145 |
+
value: 88.818
|
146 |
- type: ndcg_at_3
|
147 |
+
value: 86.16
|
148 |
- type: ndcg_at_5
|
149 |
+
value: 87.22
|
150 |
- type: precision_at_1
|
151 |
+
value: 77.766
|
152 |
- type: precision_at_10
|
153 |
+
value: 9.841999999999999
|
154 |
- type: precision_at_100
|
155 |
+
value: 1.0070000000000001
|
156 |
- type: precision_at_1000
|
157 |
value: 0.101
|
158 |
- type: precision_at_3
|
159 |
+
value: 30.875000000000004
|
160 |
- type: precision_at_5
|
161 |
+
value: 19.073
|
162 |
- type: recall_at_1
|
163 |
+
value: 77.371
|
164 |
- type: recall_at_10
|
165 |
+
value: 97.366
|
166 |
- type: recall_at_100
|
167 |
+
value: 99.684
|
168 |
- type: recall_at_1000
|
169 |
value: 100.0
|
170 |
- type: recall_at_3
|
171 |
+
value: 91.702
|
172 |
- type: recall_at_5
|
173 |
+
value: 94.31
|
174 |
- type: main_score
|
175 |
+
value: 88.237
|
176 |
task:
|
177 |
type: Retrieval
|
178 |
- dataset:
|
|
|
183 |
type: C-MTEB/DuRetrieval
|
184 |
metrics:
|
185 |
- type: map_at_1
|
186 |
+
value: 27.772000000000002
|
187 |
- type: map_at_10
|
188 |
+
value: 84.734
|
189 |
- type: map_at_100
|
190 |
+
value: 87.298
|
191 |
- type: map_at_1000
|
192 |
+
value: 87.32900000000001
|
193 |
- type: map_at_3
|
194 |
+
value: 59.431
|
195 |
- type: map_at_5
|
196 |
+
value: 74.82900000000001
|
197 |
- type: mrr_at_1
|
198 |
value: 93.65
|
199 |
- type: mrr_at_10
|
200 |
+
value: 95.568
|
201 |
- type: mrr_at_100
|
202 |
+
value: 95.608
|
203 |
- type: mrr_at_1000
|
204 |
+
value: 95.609
|
205 |
- type: mrr_at_3
|
206 |
+
value: 95.267
|
207 |
- type: mrr_at_5
|
208 |
+
value: 95.494
|
209 |
- type: ndcg_at_1
|
210 |
value: 93.65
|
211 |
- type: ndcg_at_10
|
212 |
+
value: 90.794
|
213 |
- type: ndcg_at_100
|
214 |
+
value: 92.88300000000001
|
215 |
- type: ndcg_at_1000
|
216 |
+
value: 93.144
|
217 |
- type: ndcg_at_3
|
218 |
+
value: 90.32
|
219 |
- type: ndcg_at_5
|
220 |
+
value: 89.242
|
221 |
- type: precision_at_1
|
222 |
value: 93.65
|
223 |
- type: precision_at_10
|
224 |
+
value: 42.9
|
225 |
- type: precision_at_100
|
226 |
+
value: 4.835
|
227 |
- type: precision_at_1000
|
228 |
+
value: 0.49
|
229 |
- type: precision_at_3
|
230 |
+
value: 80.85
|
231 |
- type: precision_at_5
|
232 |
+
value: 68.14
|
233 |
- type: recall_at_1
|
234 |
+
value: 27.772000000000002
|
235 |
- type: recall_at_10
|
236 |
+
value: 91.183
|
237 |
- type: recall_at_100
|
238 |
+
value: 98.219
|
239 |
- type: recall_at_1000
|
240 |
+
value: 99.55000000000001
|
241 |
- type: recall_at_3
|
242 |
+
value: 60.911
|
243 |
- type: recall_at_5
|
244 |
+
value: 78.31099999999999
|
245 |
- type: main_score
|
246 |
+
value: 90.794
|
247 |
task:
|
248 |
type: Retrieval
|
249 |
- dataset:
|
|
|
254 |
type: C-MTEB/EcomRetrieval
|
255 |
metrics:
|
256 |
- type: map_at_1
|
257 |
+
value: 54.6
|
258 |
- type: map_at_10
|
259 |
+
value: 64.742
|
260 |
- type: map_at_100
|
261 |
+
value: 65.289
|
262 |
- type: map_at_1000
|
263 |
+
value: 65.29700000000001
|
264 |
- type: map_at_3
|
265 |
+
value: 62.183
|
266 |
- type: map_at_5
|
267 |
+
value: 63.883
|
268 |
- type: mrr_at_1
|
269 |
+
value: 54.6
|
270 |
- type: mrr_at_10
|
271 |
+
value: 64.742
|
272 |
- type: mrr_at_100
|
273 |
+
value: 65.289
|
274 |
- type: mrr_at_1000
|
275 |
+
value: 65.29700000000001
|
276 |
- type: mrr_at_3
|
277 |
+
value: 62.183
|
278 |
- type: mrr_at_5
|
279 |
+
value: 63.883
|
280 |
- type: ndcg_at_1
|
281 |
+
value: 54.6
|
282 |
- type: ndcg_at_10
|
283 |
+
value: 69.719
|
284 |
- type: ndcg_at_100
|
285 |
+
value: 72.148
|
286 |
- type: ndcg_at_1000
|
287 |
+
value: 72.393
|
288 |
- type: ndcg_at_3
|
289 |
+
value: 64.606
|
290 |
- type: ndcg_at_5
|
291 |
+
value: 67.682
|
292 |
- type: precision_at_1
|
293 |
+
value: 54.6
|
294 |
- type: precision_at_10
|
295 |
+
value: 8.53
|
296 |
- type: precision_at_100
|
297 |
+
value: 0.962
|
298 |
- type: precision_at_1000
|
299 |
+
value: 0.098
|
300 |
- type: precision_at_3
|
301 |
+
value: 23.867
|
302 |
- type: precision_at_5
|
303 |
+
value: 15.82
|
304 |
- type: recall_at_1
|
305 |
+
value: 54.6
|
306 |
- type: recall_at_10
|
307 |
+
value: 85.3
|
308 |
- type: recall_at_100
|
309 |
+
value: 96.2
|
310 |
- type: recall_at_1000
|
311 |
+
value: 98.2
|
312 |
- type: recall_at_3
|
313 |
+
value: 71.6
|
314 |
- type: recall_at_5
|
315 |
+
value: 79.10000000000001
|
316 |
- type: main_score
|
317 |
+
value: 69.719
|
318 |
task:
|
319 |
type: Retrieval
|
320 |
- dataset:
|
|
|
325 |
type: C-MTEB/Mmarco-reranking
|
326 |
metrics:
|
327 |
- type: map
|
328 |
+
value: 35.30260957061897
|
329 |
- type: mrr
|
330 |
+
value: 34.098015873015875
|
331 |
- type: main_score
|
332 |
+
value: 35.30260957061897
|
333 |
task:
|
334 |
type: Reranking
|
335 |
- dataset:
|
|
|
340 |
type: C-MTEB/MMarcoRetrieval
|
341 |
metrics:
|
342 |
- type: map_at_1
|
343 |
+
value: 69.51899999999999
|
344 |
- type: map_at_10
|
345 |
+
value: 78.816
|
346 |
- type: map_at_100
|
347 |
+
value: 79.08500000000001
|
348 |
- type: map_at_1000
|
349 |
+
value: 79.091
|
350 |
- type: map_at_3
|
351 |
+
value: 76.999
|
352 |
- type: map_at_5
|
353 |
+
value: 78.194
|
354 |
- type: mrr_at_1
|
355 |
+
value: 71.80499999999999
|
356 |
- type: mrr_at_10
|
357 |
+
value: 79.29899999999999
|
358 |
- type: mrr_at_100
|
359 |
+
value: 79.532
|
360 |
- type: mrr_at_1000
|
361 |
+
value: 79.537
|
362 |
- type: mrr_at_3
|
363 |
+
value: 77.703
|
364 |
- type: mrr_at_5
|
365 |
+
value: 78.75999999999999
|
366 |
- type: ndcg_at_1
|
367 |
+
value: 71.80499999999999
|
368 |
- type: ndcg_at_10
|
369 |
+
value: 82.479
|
370 |
- type: ndcg_at_100
|
371 |
+
value: 83.611
|
372 |
- type: ndcg_at_1000
|
373 |
+
value: 83.76400000000001
|
374 |
- type: ndcg_at_3
|
375 |
+
value: 79.065
|
376 |
- type: ndcg_at_5
|
377 |
+
value: 81.092
|
378 |
- type: precision_at_1
|
379 |
+
value: 71.80499999999999
|
380 |
- type: precision_at_10
|
381 |
+
value: 9.91
|
382 |
- type: precision_at_100
|
383 |
+
value: 1.046
|
384 |
- type: precision_at_1000
|
385 |
value: 0.106
|
386 |
- type: precision_at_3
|
387 |
+
value: 29.727999999999998
|
388 |
- type: precision_at_5
|
389 |
+
value: 18.908
|
390 |
- type: recall_at_1
|
391 |
+
value: 69.51899999999999
|
392 |
- type: recall_at_10
|
393 |
+
value: 93.24
|
394 |
- type: recall_at_100
|
395 |
+
value: 98.19099999999999
|
396 |
- type: recall_at_1000
|
397 |
+
value: 99.36500000000001
|
398 |
- type: recall_at_3
|
399 |
+
value: 84.308
|
400 |
- type: recall_at_5
|
401 |
+
value: 89.119
|
402 |
- type: main_score
|
403 |
+
value: 82.479
|
404 |
task:
|
405 |
type: Retrieval
|
406 |
- dataset:
|
|
|
413 |
- type: map_at_1
|
414 |
value: 57.8
|
415 |
- type: map_at_10
|
416 |
+
value: 64.215
|
417 |
- type: map_at_100
|
418 |
+
value: 64.78
|
419 |
- type: map_at_1000
|
420 |
+
value: 64.81099999999999
|
421 |
- type: map_at_3
|
422 |
+
value: 62.64999999999999
|
423 |
- type: map_at_5
|
424 |
+
value: 63.57000000000001
|
425 |
- type: mrr_at_1
|
426 |
value: 58.099999999999994
|
427 |
- type: mrr_at_10
|
428 |
+
value: 64.371
|
429 |
- type: mrr_at_100
|
430 |
+
value: 64.936
|
431 |
- type: mrr_at_1000
|
432 |
+
value: 64.96600000000001
|
433 |
- type: mrr_at_3
|
434 |
+
value: 62.8
|
435 |
- type: mrr_at_5
|
436 |
+
value: 63.739999999999995
|
437 |
- type: ndcg_at_1
|
438 |
value: 57.8
|
439 |
- type: ndcg_at_10
|
440 |
+
value: 67.415
|
441 |
- type: ndcg_at_100
|
442 |
+
value: 70.38799999999999
|
443 |
- type: ndcg_at_1000
|
444 |
+
value: 71.229
|
445 |
- type: ndcg_at_3
|
446 |
+
value: 64.206
|
447 |
- type: ndcg_at_5
|
448 |
+
value: 65.858
|
449 |
- type: precision_at_1
|
450 |
value: 57.8
|
451 |
- type: precision_at_10
|
452 |
+
value: 7.75
|
453 |
- type: precision_at_100
|
454 |
+
value: 0.919
|
455 |
- type: precision_at_1000
|
456 |
value: 0.099
|
457 |
- type: precision_at_3
|
458 |
+
value: 22.900000000000002
|
459 |
- type: precision_at_5
|
460 |
+
value: 14.540000000000001
|
461 |
- type: recall_at_1
|
462 |
value: 57.8
|
463 |
- type: recall_at_10
|
464 |
+
value: 77.5
|
465 |
- type: recall_at_100
|
466 |
+
value: 91.9
|
467 |
- type: recall_at_1000
|
468 |
+
value: 98.6
|
469 |
- type: recall_at_3
|
470 |
+
value: 68.7
|
471 |
- type: recall_at_5
|
472 |
+
value: 72.7
|
473 |
- type: main_score
|
474 |
+
value: 67.415
|
475 |
task:
|
476 |
type: Retrieval
|
477 |
- dataset:
|
|
|
482 |
type: C-MTEB/T2Reranking
|
483 |
metrics:
|
484 |
- type: map
|
485 |
+
value: 69.06615146698508
|
486 |
- type: mrr
|
487 |
+
value: 79.7588755091294
|
488 |
- type: main_score
|
489 |
+
value: 69.06615146698508
|
490 |
task:
|
491 |
type: Reranking
|
492 |
- dataset:
|
|
|
497 |
type: C-MTEB/T2Retrieval
|
498 |
metrics:
|
499 |
- type: map_at_1
|
500 |
+
value: 28.084999999999997
|
501 |
- type: map_at_10
|
502 |
+
value: 78.583
|
503 |
- type: map_at_100
|
504 |
+
value: 82.14399999999999
|
505 |
- type: map_at_1000
|
506 |
+
value: 82.204
|
507 |
- type: map_at_3
|
508 |
+
value: 55.422000000000004
|
509 |
- type: map_at_5
|
510 |
+
value: 67.973
|
511 |
- type: mrr_at_1
|
512 |
+
value: 91.014
|
513 |
- type: mrr_at_10
|
514 |
+
value: 93.381
|
515 |
- type: mrr_at_100
|
516 |
+
value: 93.45400000000001
|
517 |
- type: mrr_at_1000
|
518 |
+
value: 93.45599999999999
|
519 |
- type: mrr_at_3
|
520 |
+
value: 92.99300000000001
|
521 |
- type: mrr_at_5
|
522 |
+
value: 93.234
|
523 |
- type: ndcg_at_1
|
524 |
+
value: 91.014
|
525 |
- type: ndcg_at_10
|
526 |
+
value: 85.931
|
527 |
- type: ndcg_at_100
|
528 |
+
value: 89.31
|
529 |
- type: ndcg_at_1000
|
530 |
+
value: 89.869
|
531 |
- type: ndcg_at_3
|
532 |
+
value: 87.348
|
533 |
- type: ndcg_at_5
|
534 |
+
value: 85.929
|
535 |
- type: precision_at_1
|
536 |
+
value: 91.014
|
537 |
- type: precision_at_10
|
538 |
+
value: 42.495
|
539 |
- type: precision_at_100
|
540 |
+
value: 5.029999999999999
|
541 |
- type: precision_at_1000
|
542 |
value: 0.516
|
543 |
- type: precision_at_3
|
544 |
+
value: 76.248
|
545 |
- type: precision_at_5
|
546 |
+
value: 63.817
|
547 |
- type: recall_at_1
|
548 |
+
value: 28.084999999999997
|
549 |
- type: recall_at_10
|
550 |
+
value: 84.88
|
551 |
- type: recall_at_100
|
552 |
+
value: 95.902
|
553 |
- type: recall_at_1000
|
554 |
+
value: 98.699
|
555 |
- type: recall_at_3
|
556 |
+
value: 57.113
|
557 |
- type: recall_at_5
|
558 |
+
value: 71.251
|
559 |
- type: main_score
|
560 |
+
value: 85.931
|
561 |
task:
|
562 |
type: Retrieval
|
563 |
- dataset:
|
|
|
568 |
type: C-MTEB/VideoRetrieval
|
569 |
metrics:
|
570 |
- type: map_at_1
|
571 |
+
value: 66.4
|
572 |
- type: map_at_10
|
573 |
+
value: 75.86
|
574 |
- type: map_at_100
|
575 |
+
value: 76.185
|
576 |
- type: map_at_1000
|
577 |
+
value: 76.188
|
578 |
- type: map_at_3
|
579 |
+
value: 74.167
|
580 |
- type: map_at_5
|
581 |
+
value: 75.187
|
582 |
- type: mrr_at_1
|
583 |
+
value: 66.4
|
584 |
- type: mrr_at_10
|
585 |
+
value: 75.86
|
586 |
- type: mrr_at_100
|
587 |
+
value: 76.185
|
588 |
- type: mrr_at_1000
|
589 |
+
value: 76.188
|
590 |
- type: mrr_at_3
|
591 |
+
value: 74.167
|
592 |
- type: mrr_at_5
|
593 |
+
value: 75.187
|
594 |
- type: ndcg_at_1
|
595 |
+
value: 66.4
|
596 |
- type: ndcg_at_10
|
597 |
+
value: 80.03099999999999
|
598 |
- type: ndcg_at_100
|
599 |
+
value: 81.459
|
600 |
- type: ndcg_at_1000
|
601 |
+
value: 81.527
|
602 |
- type: ndcg_at_3
|
603 |
+
value: 76.621
|
604 |
- type: ndcg_at_5
|
605 |
+
value: 78.446
|
606 |
- type: precision_at_1
|
607 |
+
value: 66.4
|
608 |
- type: precision_at_10
|
609 |
+
value: 9.29
|
610 |
- type: precision_at_100
|
611 |
+
value: 0.992
|
612 |
- type: precision_at_1000
|
613 |
value: 0.1
|
614 |
- type: precision_at_3
|
615 |
+
value: 27.900000000000002
|
616 |
- type: precision_at_5
|
617 |
+
value: 17.62
|
618 |
- type: recall_at_1
|
619 |
+
value: 66.4
|
620 |
- type: recall_at_10
|
621 |
+
value: 92.9
|
622 |
- type: recall_at_100
|
623 |
+
value: 99.2
|
624 |
- type: recall_at_1000
|
625 |
+
value: 99.7
|
626 |
- type: recall_at_3
|
627 |
+
value: 83.7
|
628 |
- type: recall_at_5
|
629 |
+
value: 88.1
|
630 |
- type: main_score
|
631 |
+
value: 80.03099999999999
|
632 |
task:
|
633 |
type: Retrieval
|
634 |
+
pipeline_tag: sentence-similarity
|
635 |
tags:
|
636 |
+
- sentence-transformers
|
637 |
+
- feature-extraction
|
638 |
+
- sentence-similarity
|
639 |
- mteb
|
640 |
---
|
641 |
+
-
|
642 |
+
|
643 |
+
# XYZ-embedding-zh
|
644 |
|
645 |
+
This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 1792 dimensional dense vector space and can be used for tasks like clustering or semantic search.
|
646 |
|
647 |
+
<!--- Describe your model here -->
|
648 |
|
649 |
+
## Usage (Sentence-Transformers)
|
650 |
+
|
651 |
+
Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
|
652 |
+
|
653 |
+
```
|
654 |
pip install -U sentence-transformers
|
655 |
```
|
656 |
+
|
657 |
+
Then you can use the model like this:
|
658 |
+
|
659 |
```python
|
660 |
from sentence_transformers import SentenceTransformer
|
661 |
+
sentences = ["This is an example sentence", "Each sentence is converted"]
|
662 |
|
663 |
+
model = SentenceTransformer('fangxq/XYZ-embedding-zh')
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
664 |
embeddings = model.encode(sentences)
|
665 |
+
print(embeddings)
|
666 |
+
```
|
667 |
+
|
668 |
+
|
669 |
+
|
670 |
+
## Evaluation Results
|
671 |
+
|
672 |
+
<!--- Describe how your model was evaluated -->
|
673 |
+
|
674 |
+
For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})
|
675 |
|
676 |
+
|
677 |
+
|
678 |
+
## Full Model Architecture
|
679 |
+
```
|
680 |
+
SentenceTransformer(
|
681 |
+
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
|
682 |
+
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
|
683 |
+
(2): Dense({'in_features': 1024, 'out_features': 1792, 'bias': True, 'activation_function': 'torch.nn.modules.linear.Identity'})
|
684 |
+
)
|
685 |
```
|
686 |
+
|
687 |
+
## Citing & Authors
|
688 |
+
|