Hugging Face's logo

ai-forever
/
ru-en-RoSBERTa

Feature Extraction
sentence-transformers
Safetensors
Transformers
Russian
English
roberta
mteb
Eval Results
text-embeddings-inference
Inference Endpoints
Model card Files Community
3
ai-forever commited on
Commit
f10c850
•
1 Parent(s): 5b65334

add rumteb meta

Browse files
Files changed (1) hide show
  1. README.md +1579 -0
README.md CHANGED
@@ -1,9 +1,1588 @@
1
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: mit
3
  language:
4
  - ru
5
  - en
6
  tags:
 
7
  - transformers
8
  - sentence-transformers
9
  ---
 
1
  ---
2
+ model-index:
3
+ - name: ru-en-RoSBERTa
4
+ results:
5
+ - dataset:
6
+ config: default
7
+ name: MTEB CEDRClassification (default)
8
+ revision: c0ba03d058e3e1b2f3fd20518875a4563dd12db4
9
+ split: test
10
+ type: ai-forever/cedr-classification
11
+ metrics:
12
+ - type: accuracy
13
+ value: 44.68650371944739
14
+ - type: f1
15
+ value: 40.7601061886426
16
+ - type: lrap
17
+ value: 70.69633368756747
18
+ - type: main_score
19
+ value: 44.68650371944739
20
+ task:
21
+ type: MultilabelClassification
22
+ - dataset:
23
+ config: default
24
+ name: MTEB GeoreviewClassification (default)
25
+ revision: 3765c0d1de6b7d264bc459433c45e5a75513839c
26
+ split: test
27
+ type: ai-forever/georeview-classification
28
+ metrics:
29
+ - type: accuracy
30
+ value: 49.697265625
31
+ - type: f1
32
+ value: 47.793186725286866
33
+ - type: f1_weighted
34
+ value: 47.79131720298068
35
+ - type: main_score
36
+ value: 49.697265625
37
+ task:
38
+ type: Classification
39
+ - dataset:
40
+ config: default
41
+ name: MTEB GeoreviewClusteringP2P (default)
42
+ revision: 97a313c8fc85b47f13f33e7e9a95c1ad888c7fec
43
+ split: test
44
+ type: ai-forever/georeview-clustering-p2p
45
+ metrics:
46
+ - type: main_score
47
+ value: 65.42249614873316
48
+ - type: v_measure
49
+ value: 65.42249614873316
50
+ - type: v_measure_std
51
+ value: 0.8524815312312278
52
+ task:
53
+ type: Clustering
54
+ - dataset:
55
+ config: default
56
+ name: MTEB HeadlineClassification (default)
57
+ revision: 2fe05ee6b5832cda29f2ef7aaad7b7fe6a3609eb
58
+ split: test
59
+ type: ai-forever/headline-classification
60
+ metrics:
61
+ - type: accuracy
62
+ value: 78.0029296875
63
+ - type: f1
64
+ value: 77.95151940601424
65
+ - type: f1_weighted
66
+ value: 77.95054643947716
67
+ - type: main_score
68
+ value: 78.0029296875
69
+ task:
70
+ type: Classification
71
+ - dataset:
72
+ config: default
73
+ name: MTEB InappropriatenessClassification (default)
74
+ revision: 601651fdc45ef243751676e62dd7a19f491c0285
75
+ split: test
76
+ type: ai-forever/inappropriateness-classification
77
+ metrics:
78
+ - type: accuracy
79
+ value: 61.32324218750001
80
+ - type: ap
81
+ value: 57.11029460364367
82
+ - type: ap_weighted
83
+ value: 57.11029460364367
84
+ - type: f1
85
+ value: 60.971337406307214
86
+ - type: f1_weighted
87
+ value: 60.971337406307214
88
+ - type: main_score
89
+ value: 61.32324218750001
90
+ task:
91
+ type: Classification
92
+ - dataset:
93
+ config: default
94
+ name: MTEB KinopoiskClassification (default)
95
+ revision: 5911f26666ac11af46cb9c6849d0dc80a378af24
96
+ split: test
97
+ type: ai-forever/kinopoisk-sentiment-classification
98
+ metrics:
99
+ - type: accuracy
100
+ value: 63.27333333333334
101
+ - type: f1
102
+ value: 61.007042785228116
103
+ - type: f1_weighted
104
+ value: 61.007042785228116
105
+ - type: main_score
106
+ value: 63.27333333333334
107
+ task:
108
+ type: Classification
109
+ - dataset:
110
+ config: ru
111
+ name: MTEB MIRACLReranking (ru)
112
+ revision: 6d1962c527217f8927fca80f890f14f36b2802af
113
+ split: dev
114
+ type: miracl/mmteb-miracl-reranking
115
+ metrics:
116
+ - type: MAP@1(MIRACL)
117
+ value: 30.691000000000003
118
+ - type: MAP@10(MIRACL)
119
+ value: 49.178
120
+ - type: MAP@100(MIRACL)
121
+ value: 51.225
122
+ - type: MAP@1000(MIRACL)
123
+ value: 51.225
124
+ - type: MAP@20(MIRACL)
125
+ value: 50.613
126
+ - type: MAP@3(MIRACL)
127
+ value: 42.457
128
+ - type: MAP@5(MIRACL)
129
+ value: 46.172000000000004
130
+ - type: NDCG@1(MIRACL)
131
+ value: 51.002
132
+ - type: NDCG@10(MIRACL)
133
+ value: 56.912
134
+ - type: NDCG@100(MIRACL)
135
+ value: 61.197
136
+ - type: NDCG@1000(MIRACL)
137
+ value: 61.197
138
+ - type: NDCG@20(MIRACL)
139
+ value: 59.453
140
+ - type: NDCG@3(MIRACL)
141
+ value: 51.083
142
+ - type: NDCG@5(MIRACL)
143
+ value: 53.358000000000004
144
+ - type: P@1(MIRACL)
145
+ value: 51.002
146
+ - type: P@10(MIRACL)
147
+ value: 14.852000000000002
148
+ - type: P@100(MIRACL)
149
+ value: 1.9529999999999998
150
+ - type: P@1000(MIRACL)
151
+ value: 0.19499999999999998
152
+ - type: P@20(MIRACL)
153
+ value: 8.657
154
+ - type: P@3(MIRACL)
155
+ value: 31.435000000000002
156
+ - type: P@5(MIRACL)
157
+ value: 23.608999999999998
158
+ - type: Recall@1(MIRACL)
159
+ value: 30.691000000000003
160
+ - type: Recall@10(MIRACL)
161
+ value: 67.006
162
+ - type: Recall@100(MIRACL)
163
+ value: 79.952
164
+ - type: Recall@1000(MIRACL)
165
+ value: 79.952
166
+ - type: Recall@20(MIRACL)
167
+ value: 73.811
168
+ - type: Recall@3(MIRACL)
169
+ value: 49.142
170
+ - type: Recall@5(MIRACL)
171
+ value: 57.553
172
+ - type: main_score
173
+ value: 56.912
174
+ - type: nAUC_MAP@1000_diff1(MIRACL)
175
+ value: 10.786403475779332
176
+ - type: nAUC_MAP@1000_max(MIRACL)
177
+ value: 29.477246196287275
178
+ - type: nAUC_MAP@1000_std(MIRACL)
179
+ value: 15.938834129839046
180
+ - type: nAUC_MAP@100_diff1(MIRACL)
181
+ value: 10.786403475779332
182
+ - type: nAUC_MAP@100_max(MIRACL)
183
+ value: 29.477246196287275
184
+ - type: nAUC_MAP@100_std(MIRACL)
185
+ value: 15.938834129839046
186
+ - type: nAUC_MAP@10_diff1(MIRACL)
187
+ value: 12.255091348037595
188
+ - type: nAUC_MAP@10_max(MIRACL)
189
+ value: 26.72625370045134
190
+ - type: nAUC_MAP@10_std(MIRACL)
191
+ value: 14.180071586837812
192
+ - type: nAUC_MAP@1_diff1(MIRACL)
193
+ value: 28.616487922173768
194
+ - type: nAUC_MAP@1_max(MIRACL)
195
+ value: 12.986192530664518
196
+ - type: nAUC_MAP@1_std(MIRACL)
197
+ value: 4.086145762604503
198
+ - type: nAUC_MAP@20_diff1(MIRACL)
199
+ value: 11.360341572700476
200
+ - type: nAUC_MAP@20_max(MIRACL)
201
+ value: 28.612330384153832
202
+ - type: nAUC_MAP@20_std(MIRACL)
203
+ value: 15.787480742877937
204
+ - type: nAUC_MAP@3_diff1(MIRACL)
205
+ value: 18.033783954867623
206
+ - type: nAUC_MAP@3_max(MIRACL)
207
+ value: 20.97092332905034
208
+ - type: nAUC_MAP@3_std(MIRACL)
209
+ value: 9.106058710108279
210
+ - type: nAUC_MAP@5_diff1(MIRACL)
211
+ value: 14.784231238848433
212
+ - type: nAUC_MAP@5_max(MIRACL)
213
+ value: 23.841145797143
214
+ - type: nAUC_MAP@5_std(MIRACL)
215
+ value: 11.25686258970321
216
+ - type: nAUC_NDCG@1000_diff1(MIRACL)
217
+ value: 1.4728095471561125
218
+ - type: nAUC_NDCG@1000_max(MIRACL)
219
+ value: 39.84262968697792
220
+ - type: nAUC_NDCG@1000_std(MIRACL)
221
+ value: 22.4186410243652
222
+ - type: nAUC_NDCG@100_diff1(MIRACL)
223
+ value: 1.4728095471561125
224
+ - type: nAUC_NDCG@100_max(MIRACL)
225
+ value: 39.84262968697792
226
+ - type: nAUC_NDCG@100_std(MIRACL)
227
+ value: 22.4186410243652
228
+ - type: nAUC_NDCG@10_diff1(MIRACL)
229
+ value: 5.242996478950954
230
+ - type: nAUC_NDCG@10_max(MIRACL)
231
+ value: 33.86925934510759
232
+ - type: nAUC_NDCG@10_std(MIRACL)
233
+ value: 19.457386638149625
234
+ - type: nAUC_NDCG@1_diff1(MIRACL)
235
+ value: 16.925455715967676
236
+ - type: nAUC_NDCG@1_max(MIRACL)
237
+ value: 36.72266755084653
238
+ - type: nAUC_NDCG@1_std(MIRACL)
239
+ value: 18.357456476212622
240
+ - type: nAUC_NDCG@20_diff1(MIRACL)
241
+ value: 3.361697278095995
242
+ - type: nAUC_NDCG@20_max(MIRACL)
243
+ value: 37.38923489423496
244
+ - type: nAUC_NDCG@20_std(MIRACL)
245
+ value: 22.29168372402657
246
+ - type: nAUC_NDCG@3_diff1(MIRACL)
247
+ value: 10.936904314592084
248
+ - type: nAUC_NDCG@3_max(MIRACL)
249
+ value: 30.547718047674284
250
+ - type: nAUC_NDCG@3_std(MIRACL)
251
+ value: 15.142352896765665
252
+ - type: nAUC_NDCG@5_diff1(MIRACL)
253
+ value: 8.618074920961075
254
+ - type: nAUC_NDCG@5_max(MIRACL)
255
+ value: 30.808600807482367
256
+ - type: nAUC_NDCG@5_std(MIRACL)
257
+ value: 15.793512242130051
258
+ - type: nAUC_P@1000_diff1(MIRACL)
259
+ value: -24.81839490148569
260
+ - type: nAUC_P@1000_max(MIRACL)
261
+ value: 34.16200383739091
262
+ - type: nAUC_P@1000_std(MIRACL)
263
+ value: 20.95890369662007
264
+ - type: nAUC_P@100_diff1(MIRACL)
265
+ value: -24.818394901485657
266
+ - type: nAUC_P@100_max(MIRACL)
267
+ value: 34.16200383739092
268
+ - type: nAUC_P@100_std(MIRACL)
269
+ value: 20.958903696620112
270
+ - type: nAUC_P@10_diff1(MIRACL)
271
+ value: -22.646461560750986
272
+ - type: nAUC_P@10_max(MIRACL)
273
+ value: 34.57373514819872
274
+ - type: nAUC_P@10_std(MIRACL)
275
+ value: 24.27599718176041
276
+ - type: nAUC_P@1_diff1(MIRACL)
277
+ value: 16.925455715967676
278
+ - type: nAUC_P@1_max(MIRACL)
279
+ value: 36.72266755084653
280
+ - type: nAUC_P@1_std(MIRACL)
281
+ value: 18.357456476212622
282
+ - type: nAUC_P@20_diff1(MIRACL)
283
+ value: -23.33449798384014
284
+ - type: nAUC_P@20_max(MIRACL)
285
+ value: 34.92822081787735
286
+ - type: nAUC_P@20_std(MIRACL)
287
+ value: 25.048280657629267
288
+ - type: nAUC_P@3_diff1(MIRACL)
289
+ value: -11.60659490286
290
+ - type: nAUC_P@3_max(MIRACL)
291
+ value: 38.187883056013035
292
+ - type: nAUC_P@3_std(MIRACL)
293
+ value: 21.234776997940628
294
+ - type: nAUC_P@5_diff1(MIRACL)
295
+ value: -18.86697977242918
296
+ - type: nAUC_P@5_max(MIRACL)
297
+ value: 35.6110661197626
298
+ - type: nAUC_P@5_std(MIRACL)
299
+ value: 22.11165620702996
300
+ - type: nAUC_Recall@1000_diff1(MIRACL)
301
+ value: -31.456413113303867
302
+ - type: nAUC_Recall@1000_max(MIRACL)
303
+ value: 63.785265733309636
304
+ - type: nAUC_Recall@1000_std(MIRACL)
305
+ value: 36.587933217871914
306
+ - type: nAUC_Recall@100_diff1(MIRACL)
307
+ value: -31.456413113303867
308
+ - type: nAUC_Recall@100_max(MIRACL)
309
+ value: 63.785265733309636
310
+ - type: nAUC_Recall@100_std(MIRACL)
311
+ value: 36.587933217871914
312
+ - type: nAUC_Recall@10_diff1(MIRACL)
313
+ value: -9.518740341549913
314
+ - type: nAUC_Recall@10_max(MIRACL)
315
+ value: 35.00853357699468
316
+ - type: nAUC_Recall@10_std(MIRACL)
317
+ value: 22.79313936486099
318
+ - type: nAUC_Recall@1_diff1(MIRACL)
319
+ value: 28.616487922173768
320
+ - type: nAUC_Recall@1_max(MIRACL)
321
+ value: 12.986192530664518
322
+ - type: nAUC_Recall@1_std(MIRACL)
323
+ value: 4.086145762604503
324
+ - type: nAUC_Recall@20_diff1(MIRACL)
325
+ value: -17.771143411342166
326
+ - type: nAUC_Recall@20_max(MIRACL)
327
+ value: 47.59780316487735
328
+ - type: nAUC_Recall@20_std(MIRACL)
329
+ value: 33.25494707686132
330
+ - type: nAUC_Recall@3_diff1(MIRACL)
331
+ value: 10.171226133119783
332
+ - type: nAUC_Recall@3_max(MIRACL)
333
+ value: 21.097634288680847
334
+ - type: nAUC_Recall@3_std(MIRACL)
335
+ value: 10.087211861733298
336
+ - type: nAUC_Recall@5_diff1(MIRACL)
337
+ value: 1.6868374913242932
338
+ - type: nAUC_Recall@5_max(MIRACL)
339
+ value: 25.874440474993165
340
+ - type: nAUC_Recall@5_std(MIRACL)
341
+ value: 13.46380924822079
342
+ task:
343
+ type: Reranking
344
+ - dataset:
345
+ config: ru
346
+ name: MTEB MIRACLRetrieval (ru)
347
+ revision: main
348
+ split: dev
349
+ type: miracl/mmteb-miracl
350
+ metrics:
351
+ - type: main_score
352
+ value: 53.909
353
+ - type: map_at_1
354
+ value: 24.308
355
+ - type: map_at_10
356
+ value: 43.258
357
+ - type: map_at_100
358
+ value: 46.053
359
+ - type: map_at_1000
360
+ value: 46.176
361
+ - type: map_at_20
362
+ value: 44.962
363
+ - type: map_at_3
364
+ value: 36.129
365
+ - type: map_at_5
366
+ value: 40.077
367
+ - type: mrr_at_1
368
+ value: 49.92012779552716
369
+ - type: mrr_at_10
370
+ value: 62.639554490592865
371
+ - type: mrr_at_100
372
+ value: 63.09260401526302
373
+ - type: mrr_at_1000
374
+ value: 63.10428906436666
375
+ - type: mrr_at_20
376
+ value: 62.94919151853632
377
+ - type: mrr_at_3
378
+ value: 60.15708200212997
379
+ - type: mrr_at_5
380
+ value: 61.83439829605969
381
+ - type: nauc_map_at_1000_diff1
382
+ value: 24.249990208199268
383
+ - type: nauc_map_at_1000_max
384
+ value: 25.29688440384686
385
+ - type: nauc_map_at_1000_std
386
+ value: 2.4312163206740536
387
+ - type: nauc_map_at_100_diff1
388
+ value: 24.2554939267347
389
+ - type: nauc_map_at_100_max
390
+ value: 25.25054164924535
391
+ - type: nauc_map_at_100_std
392
+ value: 2.4121726280069757
393
+ - type: nauc_map_at_10_diff1
394
+ value: 24.411765629418987
395
+ - type: nauc_map_at_10_max
396
+ value: 23.13035697774593
397
+ - type: nauc_map_at_10_std
398
+ value: -0.1673711528601927
399
+ - type: nauc_map_at_1_diff1
400
+ value: 30.55123128484441
401
+ - type: nauc_map_at_1_max
402
+ value: 13.83849108263988
403
+ - type: nauc_map_at_1_std
404
+ value: -7.087181528435525
405
+ - type: nauc_map_at_20_diff1
406
+ value: 24.125033292556417
407
+ - type: nauc_map_at_20_max
408
+ value: 24.563171125814296
409
+ - type: nauc_map_at_20_std
410
+ value: 1.266006461448722
411
+ - type: nauc_map_at_3_diff1
412
+ value: 25.71581305774253
413
+ - type: nauc_map_at_3_max
414
+ value: 18.708623514300097
415
+ - type: nauc_map_at_3_std
416
+ value: -4.772722288463871
417
+ - type: nauc_map_at_5_diff1
418
+ value: 25.352787694389097
419
+ - type: nauc_map_at_5_max
420
+ value: 20.974296353287084
421
+ - type: nauc_map_at_5_std
422
+ value: -3.4007260047029835
423
+ - type: nauc_mrr_at_1000_diff1
424
+ value: 29.492072727604622
425
+ - type: nauc_mrr_at_1000_max
426
+ value: 34.60333674990558
427
+ - type: nauc_mrr_at_1000_std
428
+ value: 11.223537361751173
429
+ - type: nauc_mrr_at_100_diff1
430
+ value: 29.47919553914885
431
+ - type: nauc_mrr_at_100_max
432
+ value: 34.618795300361995
433
+ - type: nauc_mrr_at_100_std
434
+ value: 11.243824787491663
435
+ - type: nauc_mrr_at_10_diff1
436
+ value: 29.481060608078298
437
+ - type: nauc_mrr_at_10_max
438
+ value: 34.752363175415745
439
+ - type: nauc_mrr_at_10_std
440
+ value: 10.98618160728943
441
+ - type: nauc_mrr_at_1_diff1
442
+ value: 31.81056902767142
443
+ - type: nauc_mrr_at_1_max
444
+ value: 30.351978574096773
445
+ - type: nauc_mrr_at_1_std
446
+ value: 9.735911194663025
447
+ - type: nauc_mrr_at_20_diff1
448
+ value: 29.390754002995035
449
+ - type: nauc_mrr_at_20_max
450
+ value: 34.75816984434079
451
+ - type: nauc_mrr_at_20_std
452
+ value: 11.325226515477347
453
+ - type: nauc_mrr_at_3_diff1
454
+ value: 29.948364490803186
455
+ - type: nauc_mrr_at_3_max
456
+ value: 33.973850208221556
457
+ - type: nauc_mrr_at_3_std
458
+ value: 9.988883050022485
459
+ - type: nauc_mrr_at_5_diff1
460
+ value: 29.477773016468696
461
+ - type: nauc_mrr_at_5_max
462
+ value: 34.38532892473932
463
+ - type: nauc_mrr_at_5_std
464
+ value: 10.206783034393654
465
+ - type: nauc_ndcg_at_1000_diff1
466
+ value: 24.15494700259076
467
+ - type: nauc_ndcg_at_1000_max
468
+ value: 32.367504385127035
469
+ - type: nauc_ndcg_at_1000_std
470
+ value: 10.372857487814498
471
+ - type: nauc_ndcg_at_100_diff1
472
+ value: 23.97247958991815
473
+ - type: nauc_ndcg_at_100_max
474
+ value: 32.21110774026889
475
+ - type: nauc_ndcg_at_100_std
476
+ value: 11.065328347817761
477
+ - type: nauc_ndcg_at_10_diff1
478
+ value: 24.038789867355796
479
+ - type: nauc_ndcg_at_10_max
480
+ value: 28.14682223937745
481
+ - type: nauc_ndcg_at_10_std
482
+ value: 4.518525314723316
483
+ - type: nauc_ndcg_at_1_diff1
484
+ value: 31.81056902767142
485
+ - type: nauc_ndcg_at_1_max
486
+ value: 30.351978574096773
487
+ - type: nauc_ndcg_at_1_std
488
+ value: 9.735911194663025
489
+ - type: nauc_ndcg_at_20_diff1
490
+ value: 23.157990079778138
491
+ - type: nauc_ndcg_at_20_max
492
+ value: 30.521172934621703
493
+ - type: nauc_ndcg_at_20_std
494
+ value: 7.660125728373433
495
+ - type: nauc_ndcg_at_3_diff1
496
+ value: 24.44153871615053
497
+ - type: nauc_ndcg_at_3_max
498
+ value: 27.08209732696818
499
+ - type: nauc_ndcg_at_3_std
500
+ value: 3.8766269917792537
501
+ - type: nauc_ndcg_at_5_diff1
502
+ value: 24.952468410841863
503
+ - type: nauc_ndcg_at_5_max
504
+ value: 26.29873769608537
505
+ - type: nauc_ndcg_at_5_std
506
+ value: 1.3359423751654511
507
+ - type: nauc_precision_at_1000_diff1
508
+ value: -9.104010991734798
509
+ - type: nauc_precision_at_1000_max
510
+ value: 20.36838078039637
511
+ - type: nauc_precision_at_1000_std
512
+ value: 26.889986331386297
513
+ - type: nauc_precision_at_100_diff1
514
+ value: -7.181546793298205
515
+ - type: nauc_precision_at_100_max
516
+ value: 24.32969645433586
517
+ - type: nauc_precision_at_100_std
518
+ value: 31.546209514202232
519
+ - type: nauc_precision_at_10_diff1
520
+ value: -1.0044021788494442
521
+ - type: nauc_precision_at_10_max
522
+ value: 29.37074096666726
523
+ - type: nauc_precision_at_10_std
524
+ value: 25.000959926288214
525
+ - type: nauc_precision_at_1_diff1
526
+ value: 31.81056902767142
527
+ - type: nauc_precision_at_1_max
528
+ value: 30.351978574096773
529
+ - type: nauc_precision_at_1_std
530
+ value: 9.735911194663025
531
+ - type: nauc_precision_at_20_diff1
532
+ value: -5.242529022989003
533
+ - type: nauc_precision_at_20_max
534
+ value: 28.199268120740822
535
+ - type: nauc_precision_at_20_std
536
+ value: 28.460986811065037
537
+ - type: nauc_precision_at_3_diff1
538
+ value: 9.46419634664173
539
+ - type: nauc_precision_at_3_max
540
+ value: 32.203956451949914
541
+ - type: nauc_precision_at_3_std
542
+ value: 16.4095713138301
543
+ - type: nauc_precision_at_5_diff1
544
+ value: 3.719098257572974
545
+ - type: nauc_precision_at_5_max
546
+ value: 30.53411024247047
547
+ - type: nauc_precision_at_5_std
548
+ value: 17.926227114457067
549
+ - type: nauc_recall_at_1000_diff1
550
+ value: 12.347919922311121
551
+ - type: nauc_recall_at_1000_max
552
+ value: 62.10824756167678
553
+ - type: nauc_recall_at_1000_std
554
+ value: 65.9625810682273
555
+ - type: nauc_recall_at_100_diff1
556
+ value: 11.945066948287723
557
+ - type: nauc_recall_at_100_max
558
+ value: 37.07070306829974
559
+ - type: nauc_recall_at_100_std
560
+ value: 38.76495395051901
561
+ - type: nauc_recall_at_10_diff1
562
+ value: 14.793964290237943
563
+ - type: nauc_recall_at_10_max
564
+ value: 23.170920682517334
565
+ - type: nauc_recall_at_10_std
566
+ value: 5.07461971737137
567
+ - type: nauc_recall_at_1_diff1
568
+ value: 30.55123128484441
569
+ - type: nauc_recall_at_1_max
570
+ value: 13.83849108263988
571
+ - type: nauc_recall_at_1_std
572
+ value: -7.087181528435525
573
+ - type: nauc_recall_at_20_diff1
574
+ value: 10.349310874535616
575
+ - type: nauc_recall_at_20_max
576
+ value: 27.72667852012557
577
+ - type: nauc_recall_at_20_std
578
+ value: 13.37946493360006
579
+ - type: nauc_recall_at_3_diff1
580
+ value: 20.660181561801195
581
+ - type: nauc_recall_at_3_max
582
+ value: 16.734608747226137
583
+ - type: nauc_recall_at_3_std
584
+ value: -5.887299100086449
585
+ - type: nauc_recall_at_5_diff1
586
+ value: 19.292387971699007
587
+ - type: nauc_recall_at_5_max
588
+ value: 18.151647291256193
589
+ - type: nauc_recall_at_5_std
590
+ value: -5.3874570564310895
591
+ - type: ndcg_at_1
592
+ value: 49.919999999999995
593
+ - type: ndcg_at_10
594
+ value: 53.909
595
+ - type: ndcg_at_100
596
+ value: 61.346999999999994
597
+ - type: ndcg_at_1000
598
+ value: 62.831
599
+ - type: ndcg_at_20
600
+ value: 57.44200000000001
601
+ - type: ndcg_at_3
602
+ value: 48.034
603
+ - type: ndcg_at_5
604
+ value: 50.151
605
+ - type: precision_at_1
606
+ value: 49.919999999999995
607
+ - type: precision_at_10
608
+ value: 16.206
609
+ - type: precision_at_100
610
+ value: 2.467
611
+ - type: precision_at_1000
612
+ value: 0.27499999999999997
613
+ - type: precision_at_20
614
+ value: 9.847999999999999
615
+ - type: precision_at_3
616
+ value: 33.013999999999996
617
+ - type: precision_at_5
618
+ value: 25.495
619
+ - type: recall_at_1
620
+ value: 24.308
621
+ - type: recall_at_10
622
+ value: 64.226
623
+ - type: recall_at_100
624
+ value: 88.532
625
+ - type: recall_at_1000
626
+ value: 96.702
627
+ - type: recall_at_20
628
+ value: 73.855
629
+ - type: recall_at_3
630
+ value: 43.75
631
+ - type: recall_at_5
632
+ value: 53.293
633
+ task:
634
+ type: Retrieval
635
+ - dataset:
636
+ config: ru
637
+ name: MTEB MassiveIntentClassification (ru)
638
+ revision: 4672e20407010da34463acc759c162ca9734bca6
639
+ split: test
640
+ type: mteb/amazon_massive_intent
641
+ metrics:
642
+ - type: accuracy
643
+ value: 66.96704774714189
644
+ - type: f1
645
+ value: 63.75700201120695
646
+ - type: f1_weighted
647
+ value: 65.79948352494334
648
+ - type: main_score
649
+ value: 66.96704774714189
650
+ task:
651
+ type: Classification
652
+ - dataset:
653
+ config: ru
654
+ name: MTEB MassiveIntentClassification (ru)
655
+ revision: 4672e20407010da34463acc759c162ca9734bca6
656
+ split: validation
657
+ type: mteb/amazon_massive_intent
658
+ metrics:
659
+ - type: accuracy
660
+ value: 67.1569109690113
661
+ - type: f1
662
+ value: 62.523696684003816
663
+ - type: f1_weighted
664
+ value: 66.06565157528865
665
+ - type: main_score
666
+ value: 67.1569109690113
667
+ task:
668
+ type: Classification
669
+ - dataset:
670
+ config: ru
671
+ name: MTEB MassiveScenarioClassification (ru)
672
+ revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
673
+ split: test
674
+ type: mteb/amazon_massive_scenario
675
+ metrics:
676
+ - type: accuracy
677
+ value: 71.79556153328849
678
+ - type: f1
679
+ value: 71.04798190430378
680
+ - type: f1_weighted
681
+ value: 71.11136110921589
682
+ - type: main_score
683
+ value: 71.79556153328849
684
+ task:
685
+ type: Classification
686
+ - dataset:
687
+ config: ru
688
+ name: MTEB MassiveScenarioClassification (ru)
689
+ revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
690
+ split: validation
691
+ type: mteb/amazon_massive_scenario
692
+ metrics:
693
+ - type: accuracy
694
+ value: 71.92818494835218
695
+ - type: f1
696
+ value: 70.6124989966556
697
+ - type: f1_weighted
698
+ value: 71.41072258023155
699
+ - type: main_score
700
+ value: 71.92818494835218
701
+ task:
702
+ type: Classification
703
+ - dataset:
704
+ config: default
705
+ name: MTEB RUParaPhraserSTS (default)
706
+ revision: 43265056790b8f7c59e0139acb4be0a8dad2c8f4
707
+ split: test
708
+ type: merionum/ru_paraphraser
709
+ metrics:
710
+ - type: cosine_pearson
711
+ value: 69.4312341087414
712
+ - type: cosine_spearman
713
+ value: 76.16273410937974
714
+ - type: euclidean_pearson
715
+ value: 73.59970264325928
716
+ - type: euclidean_spearman
717
+ value: 76.16273410937974
718
+ - type: main_score
719
+ value: 76.16273410937974
720
+ - type: manhattan_pearson
721
+ value: 73.63850191752708
722
+ - type: manhattan_spearman
723
+ value: 76.22156395676978
724
+ - type: pearson
725
+ value: 69.4312341087414
726
+ - type: spearman
727
+ value: 76.16273410937974
728
+ task:
729
+ type: STS
730
+ - dataset:
731
+ config: default
732
+ name: MTEB RiaNewsRetrieval (default)
733
+ revision: 82374b0bbacda6114f39ff9c5b925fa1512ca5d7
734
+ split: test
735
+ type: ai-forever/ria-news-retrieval
736
+ metrics:
737
+ - type: main_score
738
+ value: 78.864
739
+ - type: map_at_1
740
+ value: 67.61
741
+ - type: map_at_10
742
+ value: 75.44800000000001
743
+ - type: map_at_100
744
+ value: 75.73
745
+ - type: map_at_1000
746
+ value: 75.74
747
+ - type: map_at_20
748
+ value: 75.63
749
+ - type: map_at_3
750
+ value: 74.058
751
+ - type: map_at_5
752
+ value: 74.935
753
+ - type: mrr_at_1
754
+ value: 67.61
755
+ - type: mrr_at_10
756
+ value: 75.44837698412663
757
+ - type: mrr_at_100
758
+ value: 75.7296913526584
759
+ - type: mrr_at_1000
760
+ value: 75.7404584781072
761
+ - type: mrr_at_20
762
+ value: 75.62998240983255
763
+ - type: mrr_at_3
764
+ value: 74.05833333333295
765
+ - type: mrr_at_5
766
+ value: 74.93533333333274
767
+ - type: nauc_map_at_1000_diff1
768
+ value: 76.73003886073126
769
+ - type: nauc_map_at_1000_max
770
+ value: 23.880592237559313
771
+ - type: nauc_map_at_1000_std
772
+ value: -16.639489061431295
773
+ - type: nauc_map_at_100_diff1
774
+ value: 76.72565072181389
775
+ - type: nauc_map_at_100_max
776
+ value: 23.881455390102456
777
+ - type: nauc_map_at_100_std
778
+ value: -16.63176355032267
779
+ - type: nauc_map_at_10_diff1
780
+ value: 76.64273887966773
781
+ - type: nauc_map_at_10_max
782
+ value: 23.81082154251487
783
+ - type: nauc_map_at_10_std
784
+ value: -16.77740307482434
785
+ - type: nauc_map_at_1_diff1
786
+ value: 79.73607180360645
787
+ - type: nauc_map_at_1_max
788
+ value: 21.20262368559921
789
+ - type: nauc_map_at_1_std
790
+ value: -19.089796155513238
791
+ - type: nauc_map_at_20_diff1
792
+ value: 76.7030611694817
793
+ - type: nauc_map_at_20_max
794
+ value: 23.838907707504127
795
+ - type: nauc_map_at_20_std
796
+ value: -16.672743811541736
797
+ - type: nauc_map_at_3_diff1
798
+ value: 76.50523775835022
799
+ - type: nauc_map_at_3_max
800
+ value: 23.60179905501101
801
+ - type: nauc_map_at_3_std
802
+ value: -17.693757802981956
803
+ - type: nauc_map_at_5_diff1
804
+ value: 76.61576372823448
805
+ - type: nauc_map_at_5_max
806
+ value: 23.862587318336775
807
+ - type: nauc_map_at_5_std
808
+ value: -17.0437966767025
809
+ - type: nauc_mrr_at_1000_diff1
810
+ value: 76.73003886073126
811
+ - type: nauc_mrr_at_1000_max
812
+ value: 23.880592237559313
813
+ - type: nauc_mrr_at_1000_std
814
+ value: -16.639489061431295
815
+ - type: nauc_mrr_at_100_diff1
816
+ value: 76.72565072181389
817
+ - type: nauc_mrr_at_100_max
818
+ value: 23.881455390102456
819
+ - type: nauc_mrr_at_100_std
820
+ value: -16.63176355032267
821
+ - type: nauc_mrr_at_10_diff1
822
+ value: 76.64273887966773
823
+ - type: nauc_mrr_at_10_max
824
+ value: 23.81082154251487
825
+ - type: nauc_mrr_at_10_std
826
+ value: -16.77740307482434
827
+ - type: nauc_mrr_at_1_diff1
828
+ value: 79.73607180360645
829
+ - type: nauc_mrr_at_1_max
830
+ value: 21.20262368559921
831
+ - type: nauc_mrr_at_1_std
832
+ value: -19.089796155513238
833
+ - type: nauc_mrr_at_20_diff1
834
+ value: 76.7030611694817
835
+ - type: nauc_mrr_at_20_max
836
+ value: 23.838907707504127
837
+ - type: nauc_mrr_at_20_std
838
+ value: -16.672743811541736
839
+ - type: nauc_mrr_at_3_diff1
840
+ value: 76.50523775835022
841
+ - type: nauc_mrr_at_3_max
842
+ value: 23.60179905501101
843
+ - type: nauc_mrr_at_3_std
844
+ value: -17.693757802981956
845
+ - type: nauc_mrr_at_5_diff1
846
+ value: 76.61576372823448
847
+ - type: nauc_mrr_at_5_max
848
+ value: 23.862587318336775
849
+ - type: nauc_mrr_at_5_std
850
+ value: -17.0437966767025
851
+ - type: nauc_ndcg_at_1000_diff1
852
+ value: 76.016960312922
853
+ - type: nauc_ndcg_at_1000_max
854
+ value: 25.434179222015285
855
+ - type: nauc_ndcg_at_1000_std
856
+ value: -14.489226598374966
857
+ - type: nauc_ndcg_at_100_diff1
858
+ value: 75.87402195675239
859
+ - type: nauc_ndcg_at_100_max
860
+ value: 25.562687163467295
861
+ - type: nauc_ndcg_at_100_std
862
+ value: -14.165819919505346
863
+ - type: nauc_ndcg_at_10_diff1
864
+ value: 75.47305900096035
865
+ - type: nauc_ndcg_at_10_max
866
+ value: 24.9111489869184
867
+ - type: nauc_ndcg_at_10_std
868
+ value: -15.106328069022739
869
+ - type: nauc_ndcg_at_1_diff1
870
+ value: 79.73607180360645
871
+ - type: nauc_ndcg_at_1_max
872
+ value: 21.20262368559921
873
+ - type: nauc_ndcg_at_1_std
874
+ value: -19.089796155513238
875
+ - type: nauc_ndcg_at_20_diff1
876
+ value: 75.71180859144839
877
+ - type: nauc_ndcg_at_20_max
878
+ value: 25.12671193294504
879
+ - type: nauc_ndcg_at_20_std
880
+ value: -14.582900241958443
881
+ - type: nauc_ndcg_at_3_diff1
882
+ value: 75.32126900936046
883
+ - type: nauc_ndcg_at_3_max
884
+ value: 24.39543091769943
885
+ - type: nauc_ndcg_at_3_std
886
+ value: -17.183511551234538
887
+ - type: nauc_ndcg_at_5_diff1
888
+ value: 75.46170695160178
889
+ - type: nauc_ndcg_at_5_max
890
+ value: 25.001670951020937
891
+ - type: nauc_ndcg_at_5_std
892
+ value: -15.861405796419376
893
+ - type: nauc_precision_at_1000_diff1
894
+ value: 65.48397136632431
895
+ - type: nauc_precision_at_1000_max
896
+ value: 77.05533391807842
897
+ - type: nauc_precision_at_1000_std
898
+ value: 54.14509238038628
899
+ - type: nauc_precision_at_100_diff1
900
+ value: 66.6077978535527
901
+ - type: nauc_precision_at_100_max
902
+ value: 54.07639576230772
903
+ - type: nauc_precision_at_100_std
904
+ value: 28.071043659958185
905
+ - type: nauc_precision_at_10_diff1
906
+ value: 68.71592258481675
907
+ - type: nauc_precision_at_10_max
908
+ value: 31.40944055975099
909
+ - type: nauc_precision_at_10_std
910
+ value: -4.421548783271478
911
+ - type: nauc_precision_at_1_diff1
912
+ value: 79.73607180360645
913
+ - type: nauc_precision_at_1_max
914
+ value: 21.20262368559921
915
+ - type: nauc_precision_at_1_std
916
+ value: -19.089796155513238
917
+ - type: nauc_precision_at_20_diff1
918
+ value: 68.87539427047768
919
+ - type: nauc_precision_at_20_max
920
+ value: 35.602508001542176
921
+ - type: nauc_precision_at_20_std
922
+ value: 3.6366951424017184
923
+ - type: nauc_precision_at_3_diff1
924
+ value: 70.84549884977267
925
+ - type: nauc_precision_at_3_max
926
+ value: 27.35862016332144
927
+ - type: nauc_precision_at_3_std
928
+ value: -15.255203279510601
929
+ - type: nauc_precision_at_5_diff1
930
+ value: 70.27864341297163
931
+ - type: nauc_precision_at_5_max
932
+ value: 30.29162962827962
933
+ - type: nauc_precision_at_5_std
934
+ value: -10.193470309556703
935
+ - type: nauc_recall_at_1000_diff1
936
+ value: 65.48397136632475
937
+ - type: nauc_recall_at_1000_max
938
+ value: 77.05533391807865
939
+ - type: nauc_recall_at_1000_std
940
+ value: 54.14509238038722
941
+ - type: nauc_recall_at_100_diff1
942
+ value: 66.60779785355253
943
+ - type: nauc_recall_at_100_max
944
+ value: 54.07639576230805
945
+ - type: nauc_recall_at_100_std
946
+ value: 28.071043659958207
947
+ - type: nauc_recall_at_10_diff1
948
+ value: 68.71592258481655
949
+ - type: nauc_recall_at_10_max
950
+ value: 31.409440559751168
951
+ - type: nauc_recall_at_10_std
952
+ value: -4.421548783271414
953
+ - type: nauc_recall_at_1_diff1
954
+ value: 79.73607180360645
955
+ - type: nauc_recall_at_1_max
956
+ value: 21.20262368559921
957
+ - type: nauc_recall_at_1_std
958
+ value: -19.089796155513238
959
+ - type: nauc_recall_at_20_diff1
960
+ value: 68.87539427047763
961
+ - type: nauc_recall_at_20_max
962
+ value: 35.60250800154217
963
+ - type: nauc_recall_at_20_std
964
+ value: 3.6366951424018716
965
+ - type: nauc_recall_at_3_diff1
966
+ value: 70.84549884977265
967
+ - type: nauc_recall_at_3_max
968
+ value: 27.358620163321408
969
+ - type: nauc_recall_at_3_std
970
+ value: -15.255203279510626
971
+ - type: nauc_recall_at_5_diff1
972
+ value: 70.2786434129717
973
+ - type: nauc_recall_at_5_max
974
+ value: 30.291629628279733
975
+ - type: nauc_recall_at_5_std
976
+ value: -10.193470309556629
977
+ - type: ndcg_at_1
978
+ value: 67.61
979
+ - type: ndcg_at_10
980
+ value: 78.864
981
+ - type: ndcg_at_100
982
+ value: 80.211
983
+ - type: ndcg_at_1000
984
+ value: 80.50699999999999
985
+ - type: ndcg_at_20
986
+ value: 79.514
987
+ - type: ndcg_at_3
988
+ value: 76.05499999999999
989
+ - type: ndcg_at_5
990
+ value: 77.625
991
+ - type: precision_at_1
992
+ value: 67.61
993
+ - type: precision_at_10
994
+ value: 8.941
995
+ - type: precision_at_100
996
+ value: 0.9570000000000001
997
+ - type: precision_at_1000
998
+ value: 0.098
999
+ - type: precision_at_20
1000
+ value: 4.598
1001
+ - type: precision_at_3
1002
+ value: 27.267000000000003
1003
+ - type: precision_at_5
1004
+ value: 17.118
1005
+ - type: recall_at_1
1006
+ value: 67.61
1007
+ - type: recall_at_10
1008
+ value: 89.41
1009
+ - type: recall_at_100
1010
+ value: 95.67
1011
+ - type: recall_at_1000
1012
+ value: 98.02
1013
+ - type: recall_at_20
1014
+ value: 91.96
1015
+ - type: recall_at_3
1016
+ value: 81.8
1017
+ - type: recall_at_5
1018
+ value: 85.59
1019
+ task:
1020
+ type: Retrieval
1021
+ - dataset:
1022
+ config: default
1023
+ name: MTEB RuBQReranking (default)
1024
+ revision: 2e96b8f098fa4b0950fc58eacadeb31c0d0c7fa2
1025
+ split: test
1026
+ type: ai-forever/rubq-reranking
1027
+ metrics:
1028
+ - type: main_score
1029
+ value: 70.8676293869892
1030
+ - type: map
1031
+ value: 70.8676293869892
1032
+ - type: mrr
1033
+ value: 76.21519142795738
1034
+ - type: nAUC_map_diff1
1035
+ value: 37.107477549298316
1036
+ - type: nAUC_map_max
1037
+ value: 24.03175751284917
1038
+ - type: nAUC_map_std
1039
+ value: 10.543266622518289
1040
+ - type: nAUC_mrr_diff1
1041
+ value: 41.59000224211641
1042
+ - type: nAUC_mrr_max
1043
+ value: 31.06363682531277
1044
+ - type: nAUC_mrr_std
1045
+ value: 14.95221681925582
1046
+ task:
1047
+ type: Reranking
1048
+ - dataset:
1049
+ config: default
1050
+ name: MTEB RuBQRetrieval (default)
1051
+ revision: e19b6ffa60b3bc248e0b41f4cc37c26a55c2a67b
1052
+ split: test
1053
+ type: ai-forever/rubq-retrieval
1054
+ metrics:
1055
+ - type: main_score
1056
+ value: 66.77499999999999
1057
+ - type: map_at_1
1058
+ value: 38.964
1059
+ - type: map_at_10
1060
+ value: 58.679
1061
+ - type: map_at_100
1062
+ value: 59.74699999999999
1063
+ - type: map_at_1000
1064
+ value: 59.784000000000006
1065
+ - type: map_at_20
1066
+ value: 59.386
1067
+ - type: map_at_3
1068
+ value: 53.183
1069
+ - type: map_at_5
1070
+ value: 56.619
1071
+ - type: mrr_at_1
1072
+ value: 56.08747044917257
1073
+ - type: mrr_at_10
1074
+ value: 67.69477747757892
1075
+ - type: mrr_at_100
1076
+ value: 68.11028091076142
1077
+ - type: mrr_at_1000
1078
+ value: 68.12016895906572
1079
+ - type: mrr_at_20
1080
+ value: 67.99200829920431
1081
+ - type: mrr_at_3
1082
+ value: 65.40583136327825
1083
+ - type: mrr_at_5
1084
+ value: 66.86564223798278
1085
+ - type: nauc_map_at_1000_diff1
1086
+ value: 35.13932221843019
1087
+ - type: nauc_map_at_1000_max
1088
+ value: 31.603311334444573
1089
+ - type: nauc_map_at_1000_std
1090
+ value: -8.046320861408992
1091
+ - type: nauc_map_at_100_diff1
1092
+ value: 35.10777181986462
1093
+ - type: nauc_map_at_100_max
1094
+ value: 31.603059769116086
1095
+ - type: nauc_map_at_100_std
1096
+ value: -8.027533855390534
1097
+ - type: nauc_map_at_10_diff1
1098
+ value: 34.864122757362644
1099
+ - type: nauc_map_at_10_max
1100
+ value: 31.625252670171776
1101
+ - type: nauc_map_at_10_std
1102
+ value: -8.334256854154406
1103
+ - type: nauc_map_at_1_diff1
1104
+ value: 40.90418146524424
1105
+ - type: nauc_map_at_1_max
1106
+ value: 22.269308553048656
1107
+ - type: nauc_map_at_1_std
1108
+ value: -9.89932822257807
1109
+ - type: nauc_map_at_20_diff1
1110
+ value: 34.88664926631265
1111
+ - type: nauc_map_at_20_max
1112
+ value: 31.60883821879978
1113
+ - type: nauc_map_at_20_std
1114
+ value: -8.095294415067395
1115
+ - type: nauc_map_at_3_diff1
1116
+ value: 35.13227486507324
1117
+ - type: nauc_map_at_3_max
1118
+ value: 28.53848590790504
1119
+ - type: nauc_map_at_3_std
1120
+ value: -9.223288317647375
1121
+ - type: nauc_map_at_5_diff1
1122
+ value: 35.0811457266201
1123
+ - type: nauc_map_at_5_max
1124
+ value: 30.904120563551984
1125
+ - type: nauc_map_at_5_std
1126
+ value: -9.190854442617361
1127
+ - type: nauc_mrr_at_1000_diff1
1128
+ value: 43.43247399448727
1129
+ - type: nauc_mrr_at_1000_max
1130
+ value: 37.599979998251435
1131
+ - type: nauc_mrr_at_1000_std
1132
+ value: -8.461570912726742
1133
+ - type: nauc_mrr_at_100_diff1
1134
+ value: 43.42803056119293
1135
+ - type: nauc_mrr_at_100_max
1136
+ value: 37.60590141137654
1137
+ - type: nauc_mrr_at_100_std
1138
+ value: -8.456064029069271
1139
+ - type: nauc_mrr_at_10_diff1
1140
+ value: 43.34260974243939
1141
+ - type: nauc_mrr_at_10_max
1142
+ value: 37.7505248362988
1143
+ - type: nauc_mrr_at_10_std
1144
+ value: -8.4789005424329
1145
+ - type: nauc_mrr_at_1_diff1
1146
+ value: 46.8647472051038
1147
+ - type: nauc_mrr_at_1_max
1148
+ value: 34.40507832070825
1149
+ - type: nauc_mrr_at_1_std
1150
+ value: -9.148947481764475
1151
+ - type: nauc_mrr_at_20_diff1
1152
+ value: 43.37024314535158
1153
+ - type: nauc_mrr_at_20_max
1154
+ value: 37.62040185137823
1155
+ - type: nauc_mrr_at_20_std
1156
+ value: -8.497477607790167
1157
+ - type: nauc_mrr_at_3_diff1
1158
+ value: 42.980588675445404
1159
+ - type: nauc_mrr_at_3_max
1160
+ value: 37.43524263010435
1161
+ - type: nauc_mrr_at_3_std
1162
+ value: -8.698337782804687
1163
+ - type: nauc_mrr_at_5_diff1
1164
+ value: 43.224910985482765
1165
+ - type: nauc_mrr_at_5_max
1166
+ value: 38.00633132611649
1167
+ - type: nauc_mrr_at_5_std
1168
+ value: -8.554751807691591
1169
+ - type: nauc_ndcg_at_1000_diff1
1170
+ value: 36.58393000267959
1171
+ - type: nauc_ndcg_at_1000_max
1172
+ value: 34.491617466873194
1173
+ - type: nauc_ndcg_at_1000_std
1174
+ value: -6.968933918560401
1175
+ - type: nauc_ndcg_at_100_diff1
1176
+ value: 35.909285337288004
1177
+ - type: nauc_ndcg_at_100_max
1178
+ value: 34.60361766529284
1179
+ - type: nauc_ndcg_at_100_std
1180
+ value: -6.3241815724593256
1181
+ - type: nauc_ndcg_at_10_diff1
1182
+ value: 34.86940448346685
1183
+ - type: nauc_ndcg_at_10_max
1184
+ value: 34.89327996781203
1185
+ - type: nauc_ndcg_at_10_std
1186
+ value: -7.377912505502211
1187
+ - type: nauc_ndcg_at_1_diff1
1188
+ value: 47.16372543032823
1189
+ - type: nauc_ndcg_at_1_max
1190
+ value: 34.48620759685232
1191
+ - type: nauc_ndcg_at_1_std
1192
+ value: -8.881483248224074
1193
+ - type: nauc_ndcg_at_20_diff1
1194
+ value: 34.901006085701795
1195
+ - type: nauc_ndcg_at_20_max
1196
+ value: 34.766948088105174
1197
+ - type: nauc_ndcg_at_20_std
1198
+ value: -6.680375186500669
1199
+ - type: nauc_ndcg_at_3_diff1
1200
+ value: 35.16537335241684
1201
+ - type: nauc_ndcg_at_3_max
1202
+ value: 31.385279916552566
1203
+ - type: nauc_ndcg_at_3_std
1204
+ value: -8.871530629591442
1205
+ - type: nauc_ndcg_at_5_diff1
1206
+ value: 35.152664105492605
1207
+ - type: nauc_ndcg_at_5_max
1208
+ value: 33.89982336069226
1209
+ - type: nauc_ndcg_at_5_std
1210
+ value: -8.92795810387048
1211
+ - type: nauc_precision_at_1000_diff1
1212
+ value: -6.773234121047722
1213
+ - type: nauc_precision_at_1000_max
1214
+ value: 7.0059404092503925
1215
+ - type: nauc_precision_at_1000_std
1216
+ value: 4.757430160226248
1217
+ - type: nauc_precision_at_100_diff1
1218
+ value: -6.88009476644726
1219
+ - type: nauc_precision_at_100_max
1220
+ value: 10.391099419327492
1221
+ - type: nauc_precision_at_100_std
1222
+ value: 7.203837158689326
1223
+ - type: nauc_precision_at_10_diff1
1224
+ value: -0.7155570800016817
1225
+ - type: nauc_precision_at_10_max
1226
+ value: 21.06902041338105
1227
+ - type: nauc_precision_at_10_std
1228
+ value: 3.7465404459270815
1229
+ - type: nauc_precision_at_1_diff1
1230
+ value: 47.16372543032823
1231
+ - type: nauc_precision_at_1_max
1232
+ value: 34.48620759685232
1233
+ - type: nauc_precision_at_1_std
1234
+ value: -8.881483248224074
1235
+ - type: nauc_precision_at_20_diff1
1236
+ value: -4.695792117927824
1237
+ - type: nauc_precision_at_20_max
1238
+ value: 16.53698826752203
1239
+ - type: nauc_precision_at_20_std
1240
+ value: 6.681726081495262
1241
+ - type: nauc_precision_at_3_diff1
1242
+ value: 12.446292477522807
1243
+ - type: nauc_precision_at_3_max
1244
+ value: 27.622770072159884
1245
+ - type: nauc_precision_at_3_std
1246
+ value: -2.243774812074271
1247
+ - type: nauc_precision_at_5_diff1
1248
+ value: 5.851972491534291
1249
+ - type: nauc_precision_at_5_max
1250
+ value: 25.400246002612235
1251
+ - type: nauc_precision_at_5_std
1252
+ value: -0.8059534151280825
1253
+ - type: nauc_recall_at_1000_diff1
1254
+ value: 17.33619903703495
1255
+ - type: nauc_recall_at_1000_max
1256
+ value: 46.39520954734979
1257
+ - type: nauc_recall_at_1000_std
1258
+ value: 59.70020859630654
1259
+ - type: nauc_recall_at_100_diff1
1260
+ value: 9.309667388080348
1261
+ - type: nauc_recall_at_100_max
1262
+ value: 35.92482580062717
1263
+ - type: nauc_recall_at_100_std
1264
+ value: 24.021627313676188
1265
+ - type: nauc_recall_at_10_diff1
1266
+ value: 19.87959406394684
1267
+ - type: nauc_recall_at_10_max
1268
+ value: 35.00740821313158
1269
+ - type: nauc_recall_at_10_std
1270
+ value: -2.6455284599102784
1271
+ - type: nauc_recall_at_1_diff1
1272
+ value: 40.90418146524424
1273
+ - type: nauc_recall_at_1_max
1274
+ value: 22.269308553048656
1275
+ - type: nauc_recall_at_1_std
1276
+ value: -9.89932822257807
1277
+ - type: nauc_recall_at_20_diff1
1278
+ value: 15.028975252982061
1279
+ - type: nauc_recall_at_20_max
1280
+ value: 34.901307836728016
1281
+ - type: nauc_recall_at_20_std
1282
+ value: 2.9027647776175494
1283
+ - type: nauc_recall_at_3_diff1
1284
+ value: 26.13225834790859
1285
+ - type: nauc_recall_at_3_max
1286
+ value: 27.915627935543725
1287
+ - type: nauc_recall_at_3_std
1288
+ value: -8.069525359773976
1289
+ - type: nauc_recall_at_5_diff1
1290
+ value: 24.184086614024686
1291
+ - type: nauc_recall_at_5_max
1292
+ value: 32.607378848166675
1293
+ - type: nauc_recall_at_5_std
1294
+ value: -7.730984752196379
1295
+ - type: ndcg_at_1
1296
+ value: 55.969
1297
+ - type: ndcg_at_10
1298
+ value: 66.77499999999999
1299
+ - type: ndcg_at_100
1300
+ value: 70.324
1301
+ - type: ndcg_at_1000
1302
+ value: 70.95700000000001
1303
+ - type: ndcg_at_20
1304
+ value: 68.613
1305
+ - type: ndcg_at_3
1306
+ value: 59.256
1307
+ - type: ndcg_at_5
1308
+ value: 63.223
1309
+ - type: precision_at_1
1310
+ value: 55.969
1311
+ - type: precision_at_10
1312
+ value: 13.297999999999998
1313
+ - type: precision_at_100
1314
+ value: 1.585
1315
+ - type: precision_at_1000
1316
+ value: 0.167
1317
+ - type: precision_at_20
1318
+ value: 7.222
1319
+ - type: precision_at_3
1320
+ value: 32.467
1321
+ - type: precision_at_5
1322
+ value: 23.073
1323
+ - type: recall_at_1
1324
+ value: 38.964
1325
+ - type: recall_at_10
1326
+ value: 81.248
1327
+ - type: recall_at_100
1328
+ value: 95.124
1329
+ - type: recall_at_1000
1330
+ value: 99.30600000000001
1331
+ - type: recall_at_20
1332
+ value: 87.35199999999999
1333
+ - type: recall_at_3
1334
+ value: 62.785000000000004
1335
+ - type: recall_at_5
1336
+ value: 71.986
1337
+ task:
1338
+ type: Retrieval
1339
+ - dataset:
1340
+ config: default
1341
+ name: MTEB RuReviewsClassification (default)
1342
+ revision: f6d2c31f4dc6b88f468552750bfec05b4b41b05a
1343
+ split: test
1344
+ type: ai-forever/ru-reviews-classification
1345
+ metrics:
1346
+ - type: accuracy
1347
+ value: 67.958984375
1348
+ - type: f1
1349
+ value: 67.250877785427
1350
+ - type: f1_weighted
1351
+ value: 67.25215701797296
1352
+ - type: main_score
1353
+ value: 67.958984375
1354
+ task:
1355
+ type: Classification
1356
+ - dataset:
1357
+ config: default
1358
+ name: MTEB RuSTSBenchmarkSTS (default)
1359
+ revision: 7cf24f325c6da6195df55bef3d86b5e0616f3018
1360
+ split: test
1361
+ type: ai-forever/ru-stsbenchmark-sts
1362
+ metrics:
1363
+ - type: cosine_pearson
1364
+ value: 79.11336124619963
1365
+ - type: cosine_spearman
1366
+ value: 78.69157477180703
1367
+ - type: euclidean_pearson
1368
+ value: 77.84066073571212
1369
+ - type: euclidean_spearman
1370
+ value: 78.69157477180703
1371
+ - type: main_score
1372
+ value: 78.69157477180703
1373
+ - type: manhattan_pearson
1374
+ value: 77.79213012957939
1375
+ - type: manhattan_spearman
1376
+ value: 78.61384378877501
1377
+ - type: pearson
1378
+ value: 79.11336124619963
1379
+ - type: spearman
1380
+ value: 78.69157477180703
1381
+ task:
1382
+ type: STS
1383
+ - dataset:
1384
+ config: default
1385
+ name: MTEB RuSciBenchGRNTIClassification (default)
1386
+ revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
1387
+ split: test
1388
+ type: ai-forever/ru-scibench-grnti-classification
1389
+ metrics:
1390
+ - type: accuracy
1391
+ value: 59.326171875
1392
+ - type: f1
1393
+ value: 58.01171745357119
1394
+ - type: f1_weighted
1395
+ value: 58.02106511480968
1396
+ - type: main_score
1397
+ value: 59.326171875
1398
+ task:
1399
+ type: Classification
1400
+ - dataset:
1401
+ config: default
1402
+ name: MTEB RuSciBenchGRNTIClusteringP2P (default)
1403
+ revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
1404
+ split: test
1405
+ type: ai-forever/ru-scibench-grnti-classification
1406
+ metrics:
1407
+ - type: main_score
1408
+ value: 55.46570753380975
1409
+ - type: v_measure
1410
+ value: 55.46570753380975
1411
+ - type: v_measure_std
1412
+ value: 0.9813885872798612
1413
+ task:
1414
+ type: Clustering
1415
+ - dataset:
1416
+ config: default
1417
+ name: MTEB RuSciBenchOECDClassification (default)
1418
+ revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
1419
+ split: test
1420
+ type: ai-forever/ru-scibench-oecd-classification
1421
+ metrics:
1422
+ - type: accuracy
1423
+ value: 46.328125
1424
+ - type: f1
1425
+ value: 44.19158709013339
1426
+ - type: f1_weighted
1427
+ value: 44.190957945676026
1428
+ - type: main_score
1429
+ value: 46.328125
1430
+ task:
1431
+ type: Classification
1432
+ - dataset:
1433
+ config: default
1434
+ name: MTEB RuSciBenchOECDClusteringP2P (default)
1435
+ revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
1436
+ split: test
1437
+ type: ai-forever/ru-scibench-oecd-classification
1438
+ metrics:
1439
+ - type: main_score
1440
+ value: 47.28635342613908
1441
+ - type: v_measure
1442
+ value: 47.28635342613908
1443
+ - type: v_measure_std
1444
+ value: 0.7431017612993989
1445
+ task:
1446
+ type: Clustering
1447
+ - dataset:
1448
+ config: ru
1449
+ name: MTEB STS22 (ru)
1450
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
1451
+ split: test
1452
+ type: mteb/sts22-crosslingual-sts
1453
+ metrics:
1454
+ - type: cosine_pearson
1455
+ value: 63.10139371129796
1456
+ - type: cosine_spearman
1457
+ value: 67.06445400504978
1458
+ - type: euclidean_pearson
1459
+ value: 62.74563386470613
1460
+ - type: euclidean_spearman
1461
+ value: 67.06445400504978
1462
+ - type: main_score
1463
+ value: 67.06445400504978
1464
+ - type: manhattan_pearson
1465
+ value: 62.540465664732395
1466
+ - type: manhattan_spearman
1467
+ value: 66.65899492022648
1468
+ - type: pearson
1469
+ value: 63.10139371129796
1470
+ - type: spearman
1471
+ value: 67.06445400504978
1472
+ task:
1473
+ type: STS
1474
+ - dataset:
1475
+ config: default
1476
+ name: MTEB SensitiveTopicsClassification (default)
1477
+ revision: 416b34a802308eac30e4192afc0ff99bb8dcc7f2
1478
+ split: test
1479
+ type: ai-forever/sensitive-topics-classification
1480
+ metrics:
1481
+ - type: accuracy
1482
+ value: 33.0712890625
1483
+ - type: f1
1484
+ value: 38.063573562290024
1485
+ - type: lrap
1486
+ value: 49.586995442707696
1487
+ - type: main_score
1488
+ value: 33.0712890625
1489
+ task:
1490
+ type: MultilabelClassification
1491
+ - dataset:
1492
+ config: default
1493
+ name: MTEB TERRa (default)
1494
+ revision: 7b58f24536063837d644aab9a023c62199b2a612
1495
+ split: dev
1496
+ type: ai-forever/terra-pairclassification
1497
+ metrics:
1498
+ - type: cosine_accuracy
1499
+ value: 61.563517915309454
1500
+ - type: cosine_accuracy_threshold
1501
+ value: 75.3734290599823
1502
+ - type: cosine_ap
1503
+ value: 60.78861909325018
1504
+ - type: cosine_f1
1505
+ value: 67.25663716814158
1506
+ - type: cosine_f1_threshold
1507
+ value: 54.05237674713135
1508
+ - type: cosine_precision
1509
+ value: 50.836120401337794
1510
+ - type: cosine_recall
1511
+ value: 99.34640522875817
1512
+ - type: dot_accuracy
1513
+ value: 61.563517915309454
1514
+ - type: dot_accuracy_threshold
1515
+ value: 75.37343502044678
1516
+ - type: dot_ap
1517
+ value: 60.78861909325018
1518
+ - type: dot_f1
1519
+ value: 67.25663716814158
1520
+ - type: dot_f1_threshold
1521
+ value: 54.05237674713135
1522
+ - type: dot_precision
1523
+ value: 50.836120401337794
1524
+ - type: dot_recall
1525
+ value: 99.34640522875817
1526
+ - type: euclidean_accuracy
1527
+ value: 61.563517915309454
1528
+ - type: euclidean_accuracy_threshold
1529
+ value: 70.18057107925415
1530
+ - type: euclidean_ap
1531
+ value: 60.78861909325018
1532
+ - type: euclidean_f1
1533
+ value: 67.25663716814158
1534
+ - type: euclidean_f1_threshold
1535
+ value: 95.86195945739746
1536
+ - type: euclidean_precision
1537
+ value: 50.836120401337794
1538
+ - type: euclidean_recall
1539
+ value: 99.34640522875817
1540
+ - type: main_score
1541
+ value: 60.78861909325018
1542
+ - type: manhattan_accuracy
1543
+ value: 60.91205211726385
1544
+ - type: manhattan_accuracy_threshold
1545
+ value: 1813.1645202636719
1546
+ - type: manhattan_ap
1547
+ value: 60.478709337038936
1548
+ - type: manhattan_f1
1549
+ value: 67.10816777041943
1550
+ - type: manhattan_f1_threshold
1551
+ value: 2475.027275085449
1552
+ - type: manhattan_precision
1553
+ value: 50.66666666666667
1554
+ - type: manhattan_recall
1555
+ value: 99.34640522875817
1556
+ - type: max_ap
1557
+ value: 60.78861909325018
1558
+ - type: max_f1
1559
+ value: 67.25663716814158
1560
+ - type: max_precision
1561
+ value: 50.836120401337794
1562
+ - type: max_recall
1563
+ value: 99.34640522875817
1564
+ - type: similarity_accuracy
1565
+ value: 61.563517915309454
1566
+ - type: similarity_accuracy_threshold
1567
+ value: 75.3734290599823
1568
+ - type: similarity_ap
1569
+ value: 60.78861909325018
1570
+ - type: similarity_f1
1571
+ value: 67.25663716814158
1572
+ - type: similarity_f1_threshold
1573
+ value: 54.05237674713135
1574
+ - type: similarity_precision
1575
+ value: 50.836120401337794
1576
+ - type: similarity_recall
1577
+ value: 99.34640522875817
1578
+ task:
1579
+ type: PairClassification
1580
  license: mit
1581
  language:
1582
  - ru
1583
  - en
1584
  tags:
1585
+ - mteb
1586
  - transformers
1587
  - sentence-transformers
1588
  ---