Commit
•
744f300
1
Parent(s):
f67ef8e
Update README.md
Browse files
README.md
CHANGED
|
@@ -42,7 +42,7 @@ model = kenlm.Model(model_file)
|
|
| 42 |
|
| 43 |
text = "I love eating cookies !" # pre-tokenized
|
| 44 |
model.perplexity(text)
|
| 45 |
-
#
|
| 46 |
```
|
| 47 |
|
| 48 |
It is recommended to use spaCy as a preprocessor to automatically use the same tagsets and tokenization as were used when creating the LMs.
|
|
|
|
| 42 |
|
| 43 |
text = "I love eating cookies !" # pre-tokenized
|
| 44 |
model.perplexity(text)
|
| 45 |
+
# 557.3027766772162
|
| 46 |
```
|
| 47 |
|
| 48 |
It is recommended to use spaCy as a preprocessor to automatically use the same tagsets and tokenization as were used when creating the LMs.
|