Update README.md
Browse files
README.md
CHANGED
@@ -110,7 +110,11 @@ The model was trained on the text of this dataset: [https://huggingface.co/datas
|
|
110 |
**Properties:** The model was trained on the text of the fineweb-edu-llama3-annotations dataset, but with annotations coming from Nemotron-4-340B-Instruct instead of the provided annotations from Llama 3.1 70B. The dataset is a randomly sampled 467k document subset of the FineWeb dataset, which contains filtered documents crawled from the web. Please see https://arxiv.org/abs/2406.17557 for more details. <br>
|
111 |
|
112 |
### Evaluation Results
|
113 |
-
The models were shown to be useful in classifying high-quality content for LLM pretraining as part of an ensemble in the [Nemotron-CC](https://arxiv.org/abs/2412.02595) paper. See Table 9.
|
|
|
|
|
|
|
|
|
114 |
|
115 |
## Inference
|
116 |
- Engine: Python 3.10 and PyTorch
|
|
|
110 |
**Properties:** The model was trained on the text of the fineweb-edu-llama3-annotations dataset, but with annotations coming from Nemotron-4-340B-Instruct instead of the provided annotations from Llama 3.1 70B. The dataset is a randomly sampled 467k document subset of the FineWeb dataset, which contains filtered documents crawled from the web. Please see https://arxiv.org/abs/2406.17557 for more details. <br>
|
111 |
|
112 |
### Evaluation Results
|
113 |
+
The models were shown to be useful in classifying high-quality content for LLM pretraining as part of an ensemble in the [Nemotron-CC](https://arxiv.org/abs/2412.02595) paper. See Table 9 from the paper below.
|
114 |
+
|
115 |
+
<img src="https://huggingface.co/nvidia/nemocurator-fineweb-mixtral-edu-classifier/resolve/main/table_9.PNG" alt="image" style="width:750px;">
|
116 |
+
|
117 |
+
In the table above, "Ours-mistral" refers to the NemoCurator FineWeb Mixtral Edu Classifier, and "Ours-nemotron-340B" refers to the NemoCurator FineWeb Nemotron-4 Edu Classifier (this model). "Ours-ensembled" incudes the NemoCurator FineWeb Mixtral Edu Classifier, NemoCurator FineWeb Nemotron-4 Edu Classifier, and DCLM.
|
118 |
|
119 |
## Inference
|
120 |
- Engine: Python 3.10 and PyTorch
|