hotchpotch
commited on
Commit
•
cb14435
1
Parent(s):
fa1683d
Update README.md
Browse files
README.md
CHANGED
@@ -30,7 +30,7 @@ pip install yasem
|
|
30 |
```python
|
31 |
from yasem import SpladeEmbedder
|
32 |
|
33 |
-
model_name = "hotchpotch/japanese-splade-
|
34 |
embedder = SpladeEmbedder(model_name)
|
35 |
|
36 |
query = "車の燃費を向上させる方法は?"
|
@@ -139,13 +139,13 @@ tensor([
|
|
139 |
|
140 |
[JMTEB](https://github.com/sbintuitions/JMTEB) の評価結果です。
|
141 |
|
142 |
-
japanese-splade-
|
143 |
-
なお、japanese-splade-
|
144 |
|
145 |
|
146 |
| モデル名 | jagovfaqs | jaqket | mrtydi | nlp_journal <br>title_abs | nlp_journal <br>abs_intro | nlp_journal <br>title_intro | Avg <br><512 | Avg <br>ALL |
|
147 |
| ----------------------------- | --------- | ------ | ---------- | ------------------------- | ------------------------- | --------------------------- | ------------- | ------------ |
|
148 |
-
| japanese-splade-
|
149 |
| GLuCoSE-base-ja-v2 | 0.6979 | 0.6729 | 0.4186 | 0.9511 | 0.9029 | 0.7580 | 0.6851 | 0.7336 |
|
150 |
| multilingual-e5-large | 0.7030 | 0.5878 | 0.4363 | 0.9470 | 0.8600 | 0.7248 | 0.6685 | 0.7098 |
|
151 |
| ruri-large | **0.7668** | 0.6174 | 0.3803 | 0.9658 | 0.8712 | 0.7797 | 0.6826 | 0.7302 |
|
|
|
30 |
```python
|
31 |
from yasem import SpladeEmbedder
|
32 |
|
33 |
+
model_name = "hotchpotch/japanese-splade-v2"
|
34 |
embedder = SpladeEmbedder(model_name)
|
35 |
|
36 |
query = "車の燃費を向上させる方法は?"
|
|
|
139 |
|
140 |
[JMTEB](https://github.com/sbintuitions/JMTEB) の評価結果です。
|
141 |
|
142 |
+
japanese-splade-v2 は [JMTEB をスパースベクトルで評価できるように変更したコード](https://github.com/hotchpotch/JMTEB/tree/add_splade)での評価となっています。
|
143 |
+
なお、japanese-splade-v2 は JMTEB タスクである jaqket, mrtydi, jagovfaqs, nlp_jornal のデータセットのtrain,devなどのデータは **学習に利用していません**。
|
144 |
|
145 |
|
146 |
| モデル名 | jagovfaqs | jaqket | mrtydi | nlp_journal <br>title_abs | nlp_journal <br>abs_intro | nlp_journal <br>title_intro | Avg <br><512 | Avg <br>ALL |
|
147 |
| ----------------------------- | --------- | ------ | ---------- | ------------------------- | ------------------------- | --------------------------- | ------------- | ------------ |
|
148 |
+
| japanese-splade-v2 | 0.7313 | 0.6986 | **0.5106** | **0.9831** | 0.9067 | 0.8026 | **0.7309** | 0.7722 |
|
149 |
| GLuCoSE-base-ja-v2 | 0.6979 | 0.6729 | 0.4186 | 0.9511 | 0.9029 | 0.7580 | 0.6851 | 0.7336 |
|
150 |
| multilingual-e5-large | 0.7030 | 0.5878 | 0.4363 | 0.9470 | 0.8600 | 0.7248 | 0.6685 | 0.7098 |
|
151 |
| ruri-large | **0.7668** | 0.6174 | 0.3803 | 0.9658 | 0.8712 | 0.7797 | 0.6826 | 0.7302 |
|