Spaces:
Runtime error
Runtime error
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,19 +1,7 @@
|
|
| 1 |
-
---
|
| 2 |
-
title: Swahili Text Embeddings Leaderboard
|
| 3 |
-
emoji: ⚡
|
| 4 |
-
colorFrom: purple
|
| 5 |
-
colorTo: green
|
| 6 |
-
sdk: streamlit
|
| 7 |
-
sdk_version: 1.36.0
|
| 8 |
-
app_file: app.py
|
| 9 |
-
pinned: false
|
| 10 |
-
license: apache-2.0
|
| 11 |
-
---
|
| 12 |
-
|
| 13 |
# 🏆 Swahili Text Embeddings Leaderboard (STEL)
|
| 14 |
|
| 15 |
<p align="center">
|
| 16 |
-
<img src="STEL.jpg" alt="STEL" width="300" height="auto">
|
| 17 |
</p>
|
| 18 |
|
| 19 |
|
|
@@ -40,11 +28,11 @@ Explore our interactive dashboards:
|
|
| 40 |
| [MultiLinguSwahili-serengeti-E250-nli-matryoshka](https://huggingface.co/sartifyllc/MultiLinguSwahili-serengeti-E250-nli-matryoshka) | sartifyllc | Yes | serengeti-E250 | Yes | 768 | 51.3997 | 45.4011 | 77.6318 | 71.4496 | 56.4492 | 61.2038 | 63.9453 | 63.4926 | 38.6383 | 32.6575 | 77.0157 | 19.287 | 9.624 |
|
| 41 |
| [MultiLinguSwahili-bert-base-sw-cased-nli-matryoshka](https://huggingface.co/sartifyllc/MultiLinguSwahili-bert-base-sw-cased-nli-matryoshka) | sartifyllc | Yes | bert-base-sw-cased | Yes | 768 | 42.2575 | 35.4278 | 82.2461 | 64.8529 | 46.1332 | 50.5649 | 57.041 | 5.16086 | 37.6302 | 19.7916 | 68.5115 | 22.57 | 17.16 |
|
| 42 |
| [MultiLinguSwahili-mxbai-embed-large-v1-nli-matryoshka](https://huggingface.co/sartifyllc/MultiLinguSwahili-mxbai-embed-large-v1-nli-matryoshka) | sartifyllc | Yes | mxbai-embed-large-v1 | Yes | 768 | 40.0694 | 36.7914 | 81.9434 | 54.2647 | 46.7182 | 51.2206 | 57.2949 | 5.4534 | 34.5571 | 9.27166 | 70.311 | 21.831 | 11.176 |
|
| 43 |
-
| [mxbai-embed-large-v1](https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1) | mixedbread-ai | Yes | mxbai-embed-large-v1 |
|
| 44 |
-
| [bert-base-uncased-swahili](https://huggingface.co/flax-community/bert-base-uncased-swahili) | flax-community | Yes | bert-base-uncased-swahili |
|
| 45 |
| [MultiLinguSwahili-bge-small-en-v1.5-nli-matryoshka](https://huggingface.co/sartifyllc/MultiLinguSwahili-serengeti-E250-nli-matryoshka) | sartifyllc | Yes | bge-small-en-v1.5 | Yes | 256 | 36.3029 | 35.107 | 67.3486 | 54.1597 | 38.0027 | 46.8393 | 51.2305 | 5.01061 | 21.7986 | 17.8461 | 62.3059 | 21.521 | 14.465 |
|
| 46 |
-
| [bert-base-sw-cased](https://huggingface.co/Geotrend/bert-base-sw-cased) | Geotrend | Yes | bert-base-sw-cased |
|
| 47 |
-
| [UBC-NLPserengeti-E250](https://huggingface.co/UBC-NLP/serengeti-E250) | UCB | Yes | UBC-NLPserengeti-E250 |
|
| 48 |
## 🧪 Evaluation
|
| 49 |
To evaluate a model on the Swahili Embeddings Text Benchmark, you can use the following Python script:
|
| 50 |
```python
|
|
@@ -108,4 +96,4 @@ This benchmark is Swahili-based, and we need support translating and curating mo
|
|
| 108 |
|
| 109 |
---
|
| 110 |
|
| 111 |
-
Thank you for being part of this effort to advance Swahili language technologies!
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# 🏆 Swahili Text Embeddings Leaderboard (STEL)
|
| 2 |
|
| 3 |
<p align="center">
|
| 4 |
+
<img src="files/STEL.jpg" alt="STEL" width="300" height="auto">
|
| 5 |
</p>
|
| 6 |
|
| 7 |
|
|
|
|
| 28 |
| [MultiLinguSwahili-serengeti-E250-nli-matryoshka](https://huggingface.co/sartifyllc/MultiLinguSwahili-serengeti-E250-nli-matryoshka) | sartifyllc | Yes | serengeti-E250 | Yes | 768 | 51.3997 | 45.4011 | 77.6318 | 71.4496 | 56.4492 | 61.2038 | 63.9453 | 63.4926 | 38.6383 | 32.6575 | 77.0157 | 19.287 | 9.624 |
|
| 29 |
| [MultiLinguSwahili-bert-base-sw-cased-nli-matryoshka](https://huggingface.co/sartifyllc/MultiLinguSwahili-bert-base-sw-cased-nli-matryoshka) | sartifyllc | Yes | bert-base-sw-cased | Yes | 768 | 42.2575 | 35.4278 | 82.2461 | 64.8529 | 46.1332 | 50.5649 | 57.041 | 5.16086 | 37.6302 | 19.7916 | 68.5115 | 22.57 | 17.16 |
|
| 30 |
| [MultiLinguSwahili-mxbai-embed-large-v1-nli-matryoshka](https://huggingface.co/sartifyllc/MultiLinguSwahili-mxbai-embed-large-v1-nli-matryoshka) | sartifyllc | Yes | mxbai-embed-large-v1 | Yes | 768 | 40.0694 | 36.7914 | 81.9434 | 54.2647 | 46.7182 | 51.2206 | 57.2949 | 5.4534 | 34.5571 | 9.27166 | 70.311 | 21.831 | 11.176 |
|
| 31 |
+
| [mxbai-embed-large-v1](https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1) | mixedbread-ai | Yes | mxbai-embed-large-v1 | N/A | N/A | 39.6734 | 35.0802 | 83.4229 | 57.416 | 43.8635 | 47.1923 | 54.4678 | 5.87399 | 27.5669 | 21.1763 | 56.0497 | 23.742 | 20.229 |
|
| 32 |
+
| [bert-base-uncased-swahili](https://huggingface.co/flax-community/bert-base-uncased-swahili) | flax-community | Yes | bert-base-uncased-swahili | N/A | N/A | 37.8727 | 41.123 | 80.8838 | 66.0714 | 48.302 | 51.9334 | 64.2236 | 0.400601 | 18.6071 | 2.04101 | 58.9762 | 13.807 | 8.103 |
|
| 33 |
| [MultiLinguSwahili-bge-small-en-v1.5-nli-matryoshka](https://huggingface.co/sartifyllc/MultiLinguSwahili-serengeti-E250-nli-matryoshka) | sartifyllc | Yes | bge-small-en-v1.5 | Yes | 256 | 36.3029 | 35.107 | 67.3486 | 54.1597 | 38.0027 | 46.8393 | 51.2305 | 5.01061 | 21.7986 | 17.8461 | 62.3059 | 21.521 | 14.465 |
|
| 34 |
+
| [bert-base-sw-cased](https://huggingface.co/Geotrend/bert-base-sw-cased) | Geotrend | Yes | bert-base-sw-cased | N/A | N/A | 33.6552 | 35.3342 | 84.3066 | 62.3109 | 36.3685 | 38.7996 | 57.9199 | 0.396624 | 12.9566 | 6.77267 | 55.6602 | 10.077 | 2.959 |
|
| 35 |
+
| [UBC-NLPserengeti-E250](https://huggingface.co/UBC-NLP/serengeti-E250) | UCB | Yes | UBC-NLPserengeti-E250 | N/A | N/A | 33.581 | 44.0374 | 84.3848 | 42.1008 | 37.1957 | 38.2414 | 58.1592 | 12.7676 | 15.7357 | 14.7948 | 53.3967 | 2.041 | 0.117 |
|
| 36 |
## 🧪 Evaluation
|
| 37 |
To evaluate a model on the Swahili Embeddings Text Benchmark, you can use the following Python script:
|
| 38 |
```python
|
|
|
|
| 96 |
|
| 97 |
---
|
| 98 |
|
| 99 |
+
Thank you for being part of this effort to advance Swahili language technologies!
|