Update README.md
Browse files
README.md
CHANGED
@@ -1,8 +1,760 @@
|
|
1 |
---
|
2 |
-
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
4 |
---
|
5 |
|
|
|
6 |
# Model Card for Model ID
|
7 |
|
8 |
<!-- Provide a quick summary of what the model is/does. -->
|
@@ -104,7 +856,7 @@ Use the code below to get started with the model.
|
|
104 |
|
105 |
<!-- This section describes the evaluation protocols and provides the results. -->
|
106 |
|
107 |
-
### Testing Data, Factors &
|
108 |
|
109 |
#### Testing Data
|
110 |
|
@@ -118,9 +870,9 @@ Use the code below to get started with the model.
|
|
118 |
|
119 |
[More Information Needed]
|
120 |
|
121 |
-
####
|
122 |
|
123 |
-
<!-- These are the evaluation
|
124 |
|
125 |
[More Information Needed]
|
126 |
|
|
|
1 |
---
|
2 |
+
model-index:
|
3 |
+
- name: no_model_name_available
|
4 |
+
results:
|
5 |
+
- dataset:
|
6 |
+
config: en
|
7 |
+
name: MTEB MassiveScenarioClassification
|
8 |
+
revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
|
9 |
+
split: test
|
10 |
+
type: mteb/amazon_massive_scenario
|
11 |
+
metrics:
|
12 |
+
- type: accuracy
|
13 |
+
value: 0.7591123066577001
|
14 |
+
task:
|
15 |
+
type: Classification
|
16 |
+
- dataset:
|
17 |
+
config: default
|
18 |
+
name: MTEB ImdbClassification
|
19 |
+
revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
|
20 |
+
split: test
|
21 |
+
type: mteb/imdb
|
22 |
+
metrics:
|
23 |
+
- type: accuracy
|
24 |
+
value: 0.737088
|
25 |
+
task:
|
26 |
+
type: Classification
|
27 |
+
- dataset:
|
28 |
+
config: default
|
29 |
+
name: MTEB SCIDOCS
|
30 |
+
revision: f8c2fcf00f625baaa80f62ec5bd9e1fff3b8ae88
|
31 |
+
split: test
|
32 |
+
type: mteb/scidocs
|
33 |
+
metrics:
|
34 |
+
- type: ndcg_at_10
|
35 |
+
value: 0.14645
|
36 |
+
task:
|
37 |
+
type: Retrieval
|
38 |
+
- dataset:
|
39 |
+
config: default
|
40 |
+
name: MTEB BIOSSES
|
41 |
+
revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
|
42 |
+
split: test
|
43 |
+
type: mteb/biosses-sts
|
44 |
+
metrics:
|
45 |
+
- type: cosine_spearman
|
46 |
+
value: 0.7433663356966029
|
47 |
+
task:
|
48 |
+
type: STS
|
49 |
+
- dataset:
|
50 |
+
config: default
|
51 |
+
name: MTEB TwentyNewsgroupsClustering
|
52 |
+
revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
|
53 |
+
split: test
|
54 |
+
type: mteb/twentynewsgroups-clustering
|
55 |
+
metrics:
|
56 |
+
- type: v_measure
|
57 |
+
value: 0.4597050473156563
|
58 |
+
task:
|
59 |
+
type: Clustering
|
60 |
+
- dataset:
|
61 |
+
config: default
|
62 |
+
name: MTEB FEVER
|
63 |
+
revision: bea83ef9e8fb933d90a2f1d5515737465d613e12
|
64 |
+
split: test
|
65 |
+
type: mteb/fever
|
66 |
+
metrics:
|
67 |
+
- type: ndcg_at_10
|
68 |
+
value: 0.25419
|
69 |
+
task:
|
70 |
+
type: Retrieval
|
71 |
+
- dataset:
|
72 |
+
config: default
|
73 |
+
name: MTEB TwitterSemEval2015
|
74 |
+
revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
|
75 |
+
split: test
|
76 |
+
type: mteb/twittersemeval2015-pairclassification
|
77 |
+
metrics:
|
78 |
+
- type: ap
|
79 |
+
value: 0.6721556260260827
|
80 |
+
task:
|
81 |
+
type: PairClassification
|
82 |
+
- dataset:
|
83 |
+
config: default
|
84 |
+
name: MTEB NFCorpus
|
85 |
+
revision: ec0fa4fe99da2ff19ca1214b7966684033a58814
|
86 |
+
split: test
|
87 |
+
type: mteb/nfcorpus
|
88 |
+
metrics:
|
89 |
+
- type: ndcg_at_10
|
90 |
+
value: 0.23269
|
91 |
+
task:
|
92 |
+
type: Retrieval
|
93 |
+
- dataset:
|
94 |
+
config: default
|
95 |
+
name: MTEB HotpotQA
|
96 |
+
revision: ab518f4d6fcca38d87c25209f94beba119d02014
|
97 |
+
split: test
|
98 |
+
type: mteb/hotpotqa
|
99 |
+
metrics:
|
100 |
+
- type: ndcg_at_10
|
101 |
+
value: 0.29778
|
102 |
+
task:
|
103 |
+
type: Retrieval
|
104 |
+
- dataset:
|
105 |
+
config: default
|
106 |
+
name: MTEB STS15
|
107 |
+
revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
|
108 |
+
split: test
|
109 |
+
type: mteb/sts15-sts
|
110 |
+
metrics:
|
111 |
+
- type: cosine_spearman
|
112 |
+
value: 0.8236097824265283
|
113 |
+
task:
|
114 |
+
type: STS
|
115 |
+
- dataset:
|
116 |
+
config: default
|
117 |
+
name: MTEB CQADupstackMathematicaRetrieval
|
118 |
+
revision: 90fceea13679c63fe563ded68f3b6f06e50061de
|
119 |
+
split: test
|
120 |
+
type: mteb/cqadupstack-mathematica
|
121 |
+
metrics:
|
122 |
+
- type: ndcg_at_10
|
123 |
+
value: 0.1537
|
124 |
+
task:
|
125 |
+
type: Retrieval
|
126 |
+
- dataset:
|
127 |
+
config: default
|
128 |
+
name: MTEB CQADupstackEnglishRetrieval
|
129 |
+
revision: ad9991cb51e31e31e430383c75ffb2885547b5f0
|
130 |
+
split: test
|
131 |
+
type: mteb/cqadupstack-english
|
132 |
+
metrics:
|
133 |
+
- type: ndcg_at_10
|
134 |
+
value: 0.24932
|
135 |
+
task:
|
136 |
+
type: Retrieval
|
137 |
+
- dataset:
|
138 |
+
config: default
|
139 |
+
name: MTEB ArxivClusteringP2P
|
140 |
+
revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
|
141 |
+
split: test
|
142 |
+
type: mteb/arxiv-clustering-p2p
|
143 |
+
metrics:
|
144 |
+
- type: v_measure
|
145 |
+
value: 0.44358822700946515
|
146 |
+
task:
|
147 |
+
type: Clustering
|
148 |
+
- dataset:
|
149 |
+
config: default
|
150 |
+
name: MTEB FiQA2018
|
151 |
+
revision: 27a168819829fe9bcd655c2df245fb19452e8e06
|
152 |
+
split: test
|
153 |
+
type: mteb/fiqa
|
154 |
+
metrics:
|
155 |
+
- type: ndcg_at_10
|
156 |
+
value: 0.15899
|
157 |
+
task:
|
158 |
+
type: Retrieval
|
159 |
+
- dataset:
|
160 |
+
config: default
|
161 |
+
name: MTEB SciDocsRR
|
162 |
+
revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
|
163 |
+
split: test
|
164 |
+
type: mteb/scidocs-reranking
|
165 |
+
metrics:
|
166 |
+
- type: map
|
167 |
+
value: 0.7885898217382544
|
168 |
+
task:
|
169 |
+
type: Reranking
|
170 |
+
- dataset:
|
171 |
+
config: default
|
172 |
+
name: MTEB RedditClustering
|
173 |
+
revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
|
174 |
+
split: test
|
175 |
+
type: mteb/reddit-clustering
|
176 |
+
metrics:
|
177 |
+
- type: v_measure
|
178 |
+
value: 0.4966068486651516
|
179 |
+
task:
|
180 |
+
type: Clustering
|
181 |
+
- dataset:
|
182 |
+
config: default
|
183 |
+
name: MTEB Touche2020
|
184 |
+
revision: a34f9a33db75fa0cbb21bb5cfc3dae8dc8bec93f
|
185 |
+
split: test
|
186 |
+
type: mteb/touche2020
|
187 |
+
metrics:
|
188 |
+
- type: ndcg_at_10
|
189 |
+
value: 0.12018
|
190 |
+
task:
|
191 |
+
type: Retrieval
|
192 |
+
- dataset:
|
193 |
+
config: en
|
194 |
+
name: MTEB MTOPDomainClassification
|
195 |
+
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
|
196 |
+
split: test
|
197 |
+
type: mteb/mtop_domain
|
198 |
+
metrics:
|
199 |
+
- type: accuracy
|
200 |
+
value: 0.925079799361605
|
201 |
+
task:
|
202 |
+
type: Classification
|
203 |
+
- dataset:
|
204 |
+
config: default
|
205 |
+
name: MTEB Banking77Classification
|
206 |
+
revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
|
207 |
+
split: test
|
208 |
+
type: mteb/banking77
|
209 |
+
metrics:
|
210 |
+
- type: accuracy
|
211 |
+
value: 0.8135064935064935
|
212 |
+
task:
|
213 |
+
type: Classification
|
214 |
+
- dataset:
|
215 |
+
config: default
|
216 |
+
name: MTEB SummEval
|
217 |
+
revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
|
218 |
+
split: test
|
219 |
+
type: mteb/summeval
|
220 |
+
metrics:
|
221 |
+
- type: cosine_spearman
|
222 |
+
value: 0.2889324869563879
|
223 |
+
task:
|
224 |
+
type: Summarization
|
225 |
+
- dataset:
|
226 |
+
config: default
|
227 |
+
name: MTEB TwitterURLCorpus
|
228 |
+
revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
|
229 |
+
split: test
|
230 |
+
type: mteb/twitterurlcorpus-pairclassification
|
231 |
+
metrics:
|
232 |
+
- type: ap
|
233 |
+
value: 0.8389671787853494
|
234 |
+
task:
|
235 |
+
type: PairClassification
|
236 |
+
- dataset:
|
237 |
+
config: en-en
|
238 |
+
name: MTEB STS17
|
239 |
+
revision: faeb762787bd10488a50c8b5be4a3b82e411949c
|
240 |
+
split: test
|
241 |
+
type: mteb/sts17-crosslingual-sts
|
242 |
+
metrics:
|
243 |
+
- type: cosine_spearman
|
244 |
+
value: 0.8952714837442336
|
245 |
+
task:
|
246 |
+
type: STS
|
247 |
+
- dataset:
|
248 |
+
config: default
|
249 |
+
name: MTEB ArxivClusteringS2S
|
250 |
+
revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
|
251 |
+
split: test
|
252 |
+
type: mteb/arxiv-clustering-s2s
|
253 |
+
metrics:
|
254 |
+
- type: v_measure
|
255 |
+
value: 0.37137555761396807
|
256 |
+
task:
|
257 |
+
type: Clustering
|
258 |
+
- dataset:
|
259 |
+
config: default
|
260 |
+
name: MTEB CQADupstackWebmastersRetrieval
|
261 |
+
revision: 160c094312a0e1facb97e55eeddb698c0abe3571
|
262 |
+
split: test
|
263 |
+
type: mteb/cqadupstack-webmasters
|
264 |
+
metrics:
|
265 |
+
- type: ndcg_at_10
|
266 |
+
value: 0.25273
|
267 |
+
task:
|
268 |
+
type: Retrieval
|
269 |
+
- dataset:
|
270 |
+
config: en
|
271 |
+
name: MTEB AmazonCounterfactualClassification
|
272 |
+
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
|
273 |
+
split: test
|
274 |
+
type: mteb/amazon_counterfactual
|
275 |
+
metrics:
|
276 |
+
- type: accuracy
|
277 |
+
value: 0.6864179104477611
|
278 |
+
task:
|
279 |
+
type: Classification
|
280 |
+
- dataset:
|
281 |
+
config: default
|
282 |
+
name: MTEB MindSmallReranking
|
283 |
+
revision: 59042f120c80e8afa9cdbb224f67076cec0fc9a7
|
284 |
+
split: test
|
285 |
+
type: mteb/mind_small
|
286 |
+
metrics:
|
287 |
+
- type: map
|
288 |
+
value: 0.30932148624587386
|
289 |
+
task:
|
290 |
+
type: Reranking
|
291 |
+
- dataset:
|
292 |
+
config: default
|
293 |
+
name: MTEB MSMARCO
|
294 |
+
revision: c5a29a104738b98a9e76336939199e264163d4a0
|
295 |
+
split: dev
|
296 |
+
type: mteb/msmarco
|
297 |
+
metrics:
|
298 |
+
- type: ndcg_at_10
|
299 |
+
value: 0.20298
|
300 |
+
task:
|
301 |
+
type: Retrieval
|
302 |
+
- dataset:
|
303 |
+
config: default
|
304 |
+
name: MTEB STS13
|
305 |
+
revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
|
306 |
+
split: test
|
307 |
+
type: mteb/sts13-sts
|
308 |
+
metrics:
|
309 |
+
- type: cosine_spearman
|
310 |
+
value: 0.8282788944395595
|
311 |
+
task:
|
312 |
+
type: STS
|
313 |
+
- dataset:
|
314 |
+
config: default
|
315 |
+
name: MTEB StackOverflowDupQuestions
|
316 |
+
revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
|
317 |
+
split: test
|
318 |
+
type: mteb/stackoverflowdupquestions-reranking
|
319 |
+
metrics:
|
320 |
+
- type: map
|
321 |
+
value: 0.4666186087388775
|
322 |
+
task:
|
323 |
+
type: Reranking
|
324 |
+
- dataset:
|
325 |
+
config: en
|
326 |
+
name: MTEB STS22
|
327 |
+
revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
|
328 |
+
split: test
|
329 |
+
type: mteb/sts22-crosslingual-sts
|
330 |
+
metrics:
|
331 |
+
- type: cosine_spearman
|
332 |
+
value: 0.6528643554148379
|
333 |
+
task:
|
334 |
+
type: STS
|
335 |
+
- dataset:
|
336 |
+
config: en
|
337 |
+
name: MTEB MTOPIntentClassification
|
338 |
+
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
|
339 |
+
split: test
|
340 |
+
type: mteb/mtop_intent
|
341 |
+
metrics:
|
342 |
+
- type: accuracy
|
343 |
+
value: 0.6971044231646147
|
344 |
+
task:
|
345 |
+
type: Classification
|
346 |
+
- dataset:
|
347 |
+
config: default
|
348 |
+
name: MTEB RedditClusteringP2P
|
349 |
+
revision: 385e3cb46b4cfa89021f56c4380204149d0efe33
|
350 |
+
split: test
|
351 |
+
type: mteb/reddit-clustering-p2p
|
352 |
+
metrics:
|
353 |
+
- type: v_measure
|
354 |
+
value: 0.5520754883265135
|
355 |
+
task:
|
356 |
+
type: Clustering
|
357 |
+
- dataset:
|
358 |
+
config: default
|
359 |
+
name: MTEB CQADupstackUnixRetrieval
|
360 |
+
revision: 6c6430d3a6d36f8d2a829195bc5dc94d7e063e53
|
361 |
+
split: test
|
362 |
+
type: mteb/cqadupstack-unix
|
363 |
+
metrics:
|
364 |
+
- type: ndcg_at_10
|
365 |
+
value: 0.23623
|
366 |
+
task:
|
367 |
+
type: Retrieval
|
368 |
+
- dataset:
|
369 |
+
config: default
|
370 |
+
name: MTEB AskUbuntuDupQuestions
|
371 |
+
revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
|
372 |
+
split: test
|
373 |
+
type: mteb/askubuntudupquestions-reranking
|
374 |
+
metrics:
|
375 |
+
- type: map
|
376 |
+
value: 0.5611127024658868
|
377 |
+
task:
|
378 |
+
type: Reranking
|
379 |
+
- dataset:
|
380 |
+
config: default
|
381 |
+
name: MTEB QuoraRetrieval
|
382 |
+
revision: e4e08e0b7dbe3c8700f0daef558ff32256715259
|
383 |
+
split: test
|
384 |
+
type: mteb/quora
|
385 |
+
metrics:
|
386 |
+
- type: ndcg_at_10
|
387 |
+
value: 0.84318
|
388 |
+
task:
|
389 |
+
type: Retrieval
|
390 |
+
- dataset:
|
391 |
+
config: en
|
392 |
+
name: MTEB MassiveIntentClassification
|
393 |
+
revision: 4672e20407010da34463acc759c162ca9734bca6
|
394 |
+
split: test
|
395 |
+
type: mteb/amazon_massive_intent
|
396 |
+
metrics:
|
397 |
+
- type: accuracy
|
398 |
+
value: 0.7035305985205111
|
399 |
+
task:
|
400 |
+
type: Classification
|
401 |
+
- dataset:
|
402 |
+
config: default
|
403 |
+
name: MTEB SprintDuplicateQuestions
|
404 |
+
revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
|
405 |
+
split: test
|
406 |
+
type: mteb/sprintduplicatequestions-pairclassification
|
407 |
+
metrics:
|
408 |
+
- type: ap
|
409 |
+
value: 0.8547019640605763
|
410 |
+
task:
|
411 |
+
type: PairClassification
|
412 |
+
- dataset:
|
413 |
+
config: default
|
414 |
+
name: MTEB AmazonPolarityClassification
|
415 |
+
revision: e2d317d38cd51312af73b3d32a06d1a08b442046
|
416 |
+
split: test
|
417 |
+
type: mteb/amazon_polarity
|
418 |
+
metrics:
|
419 |
+
- type: accuracy
|
420 |
+
value: 0.8622175000000001
|
421 |
+
task:
|
422 |
+
type: Classification
|
423 |
+
- dataset:
|
424 |
+
config: default
|
425 |
+
name: MTEB SICK-R
|
426 |
+
revision: 20a6d6f312dd54037fe07a32d58e5e168867909d
|
427 |
+
split: test
|
428 |
+
type: mteb/sickr-sts
|
429 |
+
metrics:
|
430 |
+
- type: cosine_spearman
|
431 |
+
value: 0.8074931072126776
|
432 |
+
task:
|
433 |
+
type: STS
|
434 |
+
- dataset:
|
435 |
+
config: default
|
436 |
+
name: MTEB StackExchangeClustering
|
437 |
+
revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
|
438 |
+
split: test
|
439 |
+
type: mteb/stackexchange-clustering
|
440 |
+
metrics:
|
441 |
+
- type: v_measure
|
442 |
+
value: 0.5824562550961052
|
443 |
+
task:
|
444 |
+
type: Clustering
|
445 |
+
- dataset:
|
446 |
+
config: default
|
447 |
+
name: MTEB DBPedia
|
448 |
+
revision: c0f706b76e590d620bd6618b3ca8efdd34e2d659
|
449 |
+
split: test
|
450 |
+
type: mteb/dbpedia
|
451 |
+
metrics:
|
452 |
+
- type: ndcg_at_10
|
453 |
+
value: 0.27904
|
454 |
+
task:
|
455 |
+
type: Retrieval
|
456 |
+
- dataset:
|
457 |
+
config: default
|
458 |
+
name: MTEB CQADupstackProgrammersRetrieval
|
459 |
+
revision: 6184bc1440d2dbc7612be22b50686b8826d22b32
|
460 |
+
split: test
|
461 |
+
type: mteb/cqadupstack-programmers
|
462 |
+
metrics:
|
463 |
+
- type: ndcg_at_10
|
464 |
+
value: 0.22149
|
465 |
+
task:
|
466 |
+
type: Retrieval
|
467 |
+
- dataset:
|
468 |
+
config: default
|
469 |
+
name: MTEB MedrxivClusteringP2P
|
470 |
+
revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
|
471 |
+
split: test
|
472 |
+
type: mteb/medrxiv-clustering-p2p
|
473 |
+
metrics:
|
474 |
+
- type: v_measure
|
475 |
+
value: 0.3237094363112699
|
476 |
+
task:
|
477 |
+
type: Clustering
|
478 |
+
- dataset:
|
479 |
+
config: default
|
480 |
+
name: MTEB ToxicConversationsClassification
|
481 |
+
revision: edfaf9da55d3dd50d43143d90c1ac476895ae6de
|
482 |
+
split: test
|
483 |
+
type: mteb/toxic_conversations_50k
|
484 |
+
metrics:
|
485 |
+
- type: accuracy
|
486 |
+
value: 0.69345703125
|
487 |
+
task:
|
488 |
+
type: Classification
|
489 |
+
- dataset:
|
490 |
+
config: default
|
491 |
+
name: MTEB CQADupstackStatsRetrieval
|
492 |
+
revision: 65ac3a16b8e91f9cee4c9828cc7c335575432a2a
|
493 |
+
split: test
|
494 |
+
type: mteb/cqadupstack-stats
|
495 |
+
metrics:
|
496 |
+
- type: ndcg_at_10
|
497 |
+
value: 0.20067
|
498 |
+
task:
|
499 |
+
type: Retrieval
|
500 |
+
- dataset:
|
501 |
+
config: default
|
502 |
+
name: MTEB STS16
|
503 |
+
revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
|
504 |
+
split: test
|
505 |
+
type: mteb/sts16-sts
|
506 |
+
metrics:
|
507 |
+
- type: cosine_spearman
|
508 |
+
value: 0.8059731462041985
|
509 |
+
task:
|
510 |
+
type: STS
|
511 |
+
- dataset:
|
512 |
+
config: default
|
513 |
+
name: MTEB SciFact
|
514 |
+
revision: 0228b52cf27578f30900b9e5271d331663a030d7
|
515 |
+
split: test
|
516 |
+
type: mteb/scifact
|
517 |
+
metrics:
|
518 |
+
- type: ndcg_at_10
|
519 |
+
value: 0.50544
|
520 |
+
task:
|
521 |
+
type: Retrieval
|
522 |
+
- dataset:
|
523 |
+
config: default
|
524 |
+
name: MTEB TweetSentimentExtractionClassification
|
525 |
+
revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
|
526 |
+
split: test
|
527 |
+
type: mteb/tweet_sentiment_extraction
|
528 |
+
metrics:
|
529 |
+
- type: accuracy
|
530 |
+
value: 0.6107243916242219
|
531 |
+
task:
|
532 |
+
type: Classification
|
533 |
+
- dataset:
|
534 |
+
config: default
|
535 |
+
name: MTEB STS12
|
536 |
+
revision: a0d554a64d88156834ff5ae9920b964011b16384
|
537 |
+
split: test
|
538 |
+
type: mteb/sts12-sts
|
539 |
+
metrics:
|
540 |
+
- type: cosine_spearman
|
541 |
+
value: 0.7407164757075673
|
542 |
+
task:
|
543 |
+
type: STS
|
544 |
+
- dataset:
|
545 |
+
config: default
|
546 |
+
name: MTEB CQADupstackPhysicsRetrieval
|
547 |
+
revision: 79531abbd1fb92d06c6d6315a0cbbbf5bb247ea4
|
548 |
+
split: test
|
549 |
+
type: mteb/cqadupstack-physics
|
550 |
+
metrics:
|
551 |
+
- type: ndcg_at_10
|
552 |
+
value: 0.31707
|
553 |
+
task:
|
554 |
+
type: Retrieval
|
555 |
+
- dataset:
|
556 |
+
config: default
|
557 |
+
name: MTEB BiorxivClusteringP2P
|
558 |
+
revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
|
559 |
+
split: test
|
560 |
+
type: mteb/biorxiv-clustering-p2p
|
561 |
+
metrics:
|
562 |
+
- type: v_measure
|
563 |
+
value: 0.36100815785085566
|
564 |
+
task:
|
565 |
+
type: Clustering
|
566 |
+
- dataset:
|
567 |
+
config: default
|
568 |
+
name: MTEB CQADupstackGisRetrieval
|
569 |
+
revision: 5003b3064772da1887988e05400cf3806fe491f2
|
570 |
+
split: test
|
571 |
+
type: mteb/cqadupstack-gis
|
572 |
+
metrics:
|
573 |
+
- type: ndcg_at_10
|
574 |
+
value: 0.22124
|
575 |
+
task:
|
576 |
+
type: Retrieval
|
577 |
+
- dataset:
|
578 |
+
config: default
|
579 |
+
name: MTEB CQADupstackGamingRetrieval
|
580 |
+
revision: 4885aa143210c98657558c04aaf3dc47cfb54340
|
581 |
+
split: test
|
582 |
+
type: mteb/cqadupstack-gaming
|
583 |
+
metrics:
|
584 |
+
- type: ndcg_at_10
|
585 |
+
value: 0.42459
|
586 |
+
task:
|
587 |
+
type: Retrieval
|
588 |
+
- dataset:
|
589 |
+
config: default
|
590 |
+
name: MTEB STSBenchmark
|
591 |
+
revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
|
592 |
+
split: test
|
593 |
+
type: mteb/stsbenchmark-sts
|
594 |
+
metrics:
|
595 |
+
- type: cosine_spearman
|
596 |
+
value: 0.7980073939799789
|
597 |
+
task:
|
598 |
+
type: STS
|
599 |
+
- dataset:
|
600 |
+
config: default
|
601 |
+
name: MTEB CQADupstackWordpressRetrieval
|
602 |
+
revision: 4ffe81d471b1924886b33c7567bfb200e9eec5c4
|
603 |
+
split: test
|
604 |
+
type: mteb/cqadupstack-wordpress
|
605 |
+
metrics:
|
606 |
+
- type: ndcg_at_10
|
607 |
+
value: 0.15801
|
608 |
+
task:
|
609 |
+
type: Retrieval
|
610 |
+
- dataset:
|
611 |
+
config: default
|
612 |
+
name: MTEB StackExchangeClusteringP2P
|
613 |
+
revision: 815ca46b2622cec33ccafc3735d572c266efdb44
|
614 |
+
split: test
|
615 |
+
type: mteb/stackexchange-clustering-p2p
|
616 |
+
metrics:
|
617 |
+
- type: v_measure
|
618 |
+
value: 0.30348330076332997
|
619 |
+
task:
|
620 |
+
type: Clustering
|
621 |
+
- dataset:
|
622 |
+
config: default
|
623 |
+
name: MTEB BiorxivClusteringS2S
|
624 |
+
revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
|
625 |
+
split: test
|
626 |
+
type: mteb/biorxiv-clustering-s2s
|
627 |
+
metrics:
|
628 |
+
- type: v_measure
|
629 |
+
value: 0.3066700709222508
|
630 |
+
task:
|
631 |
+
type: Clustering
|
632 |
+
- dataset:
|
633 |
+
config: default
|
634 |
+
name: MTEB CQADupstackTexRetrieval
|
635 |
+
revision: 46989137a86843e03a6195de44b09deda022eec7
|
636 |
+
split: test
|
637 |
+
type: mteb/cqadupstack-tex
|
638 |
+
metrics:
|
639 |
+
- type: ndcg_at_10
|
640 |
+
value: 0.14452
|
641 |
+
task:
|
642 |
+
type: Retrieval
|
643 |
+
- dataset:
|
644 |
+
config: default
|
645 |
+
name: MTEB CQADupstackAndroidRetrieval
|
646 |
+
revision: f46a197baaae43b4f621051089b82a364682dfeb
|
647 |
+
split: test
|
648 |
+
type: mteb/cqadupstack-android
|
649 |
+
metrics:
|
650 |
+
- type: ndcg_at_10
|
651 |
+
value: 0.34255
|
652 |
+
task:
|
653 |
+
type: Retrieval
|
654 |
+
- dataset:
|
655 |
+
config: default
|
656 |
+
name: MTEB TRECCOVID
|
657 |
+
revision: bb9466bac8153a0349341eb1b22e06409e78ef4e
|
658 |
+
split: test
|
659 |
+
type: mteb/trec-covid
|
660 |
+
metrics:
|
661 |
+
- type: ndcg_at_10
|
662 |
+
value: 0.37473
|
663 |
+
task:
|
664 |
+
type: Retrieval
|
665 |
+
- dataset:
|
666 |
+
config: default
|
667 |
+
name: MTEB NQ
|
668 |
+
revision: b774495ed302d8c44a3a7ea25c90dbce03968f31
|
669 |
+
split: test
|
670 |
+
type: mteb/nq
|
671 |
+
metrics:
|
672 |
+
- type: ndcg_at_10
|
673 |
+
value: 0.21214
|
674 |
+
task:
|
675 |
+
type: Retrieval
|
676 |
+
- dataset:
|
677 |
+
config: default
|
678 |
+
name: MTEB CQADupstackWebmastersRetrieval
|
679 |
+
revision: 160c094312a0e1facb97e55eeddb698c0abe3571
|
680 |
+
split: test
|
681 |
+
type: mteb/cqadupstack-webmasters
|
682 |
+
metrics:
|
683 |
+
- type: ndcg_at_10
|
684 |
+
value: 0.25273
|
685 |
+
task:
|
686 |
+
type: Retrieval
|
687 |
+
- dataset:
|
688 |
+
config: default
|
689 |
+
name: MTEB ArguAna
|
690 |
+
revision: c22ab2a51041ffd869aaddef7af8d8215647e41a
|
691 |
+
split: test
|
692 |
+
type: mteb/arguana
|
693 |
+
metrics:
|
694 |
+
- type: ndcg_at_10
|
695 |
+
value: 0.44416
|
696 |
+
task:
|
697 |
+
type: Retrieval
|
698 |
+
- dataset:
|
699 |
+
config: default
|
700 |
+
name: MTEB STS14
|
701 |
+
revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
|
702 |
+
split: test
|
703 |
+
type: mteb/sts14-sts
|
704 |
+
metrics:
|
705 |
+
- type: cosine_spearman
|
706 |
+
value: 0.7544709602964104
|
707 |
+
task:
|
708 |
+
type: STS
|
709 |
+
- dataset:
|
710 |
+
config: default
|
711 |
+
name: MTEB MedrxivClusteringS2S
|
712 |
+
revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
|
713 |
+
split: test
|
714 |
+
type: mteb/medrxiv-clustering-s2s
|
715 |
+
metrics:
|
716 |
+
- type: v_measure
|
717 |
+
value: 0.2932525018378166
|
718 |
+
task:
|
719 |
+
type: Clustering
|
720 |
+
- dataset:
|
721 |
+
config: en
|
722 |
+
name: MTEB AmazonReviewsClassification
|
723 |
+
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
|
724 |
+
split: test
|
725 |
+
type: mteb/amazon_reviews_multi
|
726 |
+
metrics:
|
727 |
+
- type: accuracy
|
728 |
+
value: 0.43157999999999996
|
729 |
+
task:
|
730 |
+
type: Classification
|
731 |
+
- dataset:
|
732 |
+
config: default
|
733 |
+
name: MTEB ClimateFEVER
|
734 |
+
revision: 47f2ac6acb640fc46020b02a5b59fdda04d39380
|
735 |
+
split: test
|
736 |
+
type: mteb/climate-fever
|
737 |
+
metrics:
|
738 |
+
- type: ndcg_at_10
|
739 |
+
value: 0.11327
|
740 |
+
task:
|
741 |
+
type: Retrieval
|
742 |
+
- dataset:
|
743 |
+
config: default
|
744 |
+
name: MTEB EmotionClassification
|
745 |
+
revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
|
746 |
+
split: test
|
747 |
+
type: mteb/emotion
|
748 |
+
metrics:
|
749 |
+
- type: accuracy
|
750 |
+
value: 0.48450000000000004
|
751 |
+
task:
|
752 |
+
type: Classification
|
753 |
+
tags:
|
754 |
+
- mteb
|
755 |
---
|
756 |
|
757 |
+
|
758 |
# Model Card for Model ID
|
759 |
|
760 |
<!-- Provide a quick summary of what the model is/does. -->
|
|
|
856 |
|
857 |
<!-- This section describes the evaluation protocols and provides the results. -->
|
858 |
|
859 |
+
### Testing Data, Factors & metricss
|
860 |
|
861 |
#### Testing Data
|
862 |
|
|
|
870 |
|
871 |
[More Information Needed]
|
872 |
|
873 |
+
#### metricss
|
874 |
|
875 |
+
<!-- These are the evaluation metricss being used, ideally with a description of why. -->
|
876 |
|
877 |
[More Information Needed]
|
878 |
|