Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -4,6 +4,7 @@ license: apache-2.0
|
|
| 4 |
|
| 5 |
Upstage `solar-pro` tokenizer
|
| 6 |
- Vocab size: 64,000
|
|
|
|
| 7 |
|
| 8 |
Please use this tokenizer for tokenizing inputs for the Upstage [solar-pro](https://developers.upstage.ai/docs/apis/chat) model.
|
| 9 |
|
|
|
|
| 4 |
|
| 5 |
Upstage `solar-pro` tokenizer
|
| 6 |
- Vocab size: 64,000
|
| 7 |
+
- Langauge support: English, Korean and more
|
| 8 |
|
| 9 |
Please use this tokenizer for tokenizing inputs for the Upstage [solar-pro](https://developers.upstage.ai/docs/apis/chat) model.
|
| 10 |
|