Update README.md
Browse files
README.md
CHANGED
@@ -10,8 +10,10 @@ inference: false
|
|
10 |
|
11 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/60e50ce5350d181892d5a636/mC3xwgJJ139R9LXbXpBWj.png)
|
12 |
|
|
|
|
|
13 |
This educational value classifier is deeply inspired by [Textbooks Are All You Need](https://arxiv.org/abs/2306.11644), where a classifier was developed to predict the educational value of data, and was then used for data filtering.
|
14 |
-
Model is built on fasttext - it can classify more than 2000 examples per second in CPU, and so
|
15 |
This model can classify if a text has high educational value (more explicitly defined then textbook quality). This definition change is a substantial change vs [kenhktsui/llm-data-textbook-quality-fasttext-classifer-v1](https://huggingface.co/kenhktsui/llm-data-textbook-quality-fasttext-classifer-v1).
|
16 |
It can be used as a filter for data curation when training a LLM.
|
17 |
There are 3 labels instead of 2 labels, as it offers higher granularity of educational value.
|
|
|
10 |
|
11 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/60e50ce5350d181892d5a636/mC3xwgJJ139R9LXbXpBWj.png)
|
12 |
|
13 |
+
## **"Garbage in, garbage out. A language model is only as good as its training data irrespective of its parameter count."**
|
14 |
+
|
15 |
This educational value classifier is deeply inspired by [Textbooks Are All You Need](https://arxiv.org/abs/2306.11644), where a classifier was developed to predict the educational value of data, and was then used for data filtering.
|
16 |
+
Model is built on fasttext - it can classify more than 2000 examples per second in CPU, and so it can be used **on-the-fly** during pretraining.
|
17 |
This model can classify if a text has high educational value (more explicitly defined then textbook quality). This definition change is a substantial change vs [kenhktsui/llm-data-textbook-quality-fasttext-classifer-v1](https://huggingface.co/kenhktsui/llm-data-textbook-quality-fasttext-classifer-v1).
|
18 |
It can be used as a filter for data curation when training a LLM.
|
19 |
There are 3 labels instead of 2 labels, as it offers higher granularity of educational value.
|