AlexHung29629 commited on
Commit
425baeb
1 Parent(s): ed6854b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -4
README.md CHANGED
@@ -1,13 +1,15 @@
1
  ---
2
- language:
3
- - ja
 
 
4
  tags:
5
  - kenlm
6
  - perplexity
7
  - n-gram
8
  - kneser-ney
9
  - bigscience
10
- license: "mit"
11
  datasets:
12
  - wikipedia
13
  ---
@@ -42,4 +44,4 @@ model.get_perplexity("I am very perplexed")
42
  model.get_perplexity("im hella trippin")
43
  # 46793.5 (high perplexity, since the sentence is colloquial and contains grammar mistakes)
44
  ```
45
- In the example above we see that, since Wikipedia is a collection of encyclopedic articles, a KenLM model trained on it will naturally give lower perplexity scores to sentences with formal language and no grammar mistakes than colloquial sentences with grammar mistakes.
 
1
  ---
2
+ language:
3
+ - ja
4
+ - de
5
+ - ru
6
  tags:
7
  - kenlm
8
  - perplexity
9
  - n-gram
10
  - kneser-ney
11
  - bigscience
12
+ license: mit
13
  datasets:
14
  - wikipedia
15
  ---
 
44
  model.get_perplexity("im hella trippin")
45
  # 46793.5 (high perplexity, since the sentence is colloquial and contains grammar mistakes)
46
  ```
47
+ In the example above we see that, since Wikipedia is a collection of encyclopedic articles, a KenLM model trained on it will naturally give lower perplexity scores to sentences with formal language and no grammar mistakes than colloquial sentences with grammar mistakes.