Safetensors
bert
sarahyurick commited on
Commit
bed4779
·
verified ·
1 Parent(s): 0d30edb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -110,7 +110,11 @@ The model was trained on the text of this dataset: [https://huggingface.co/datas
110
  **Properties:** The model was trained on the text of the fineweb-edu-llama3-annotations dataset, but with annotations coming from Nemotron-4-340B-Instruct instead of the provided annotations from Llama 3.1 70B. The dataset is a randomly sampled 467k document subset of the FineWeb dataset, which contains filtered documents crawled from the web. Please see https://arxiv.org/abs/2406.17557 for more details. <br>
111
 
112
  ### Evaluation Results
113
- The models were shown to be useful in classifying high-quality content for LLM pretraining as part of an ensemble in the [Nemotron-CC](https://arxiv.org/abs/2412.02595) paper. See Table 9.
 
 
 
 
114
 
115
  ## Inference
116
  - Engine: Python 3.10 and PyTorch
 
110
  **Properties:** The model was trained on the text of the fineweb-edu-llama3-annotations dataset, but with annotations coming from Nemotron-4-340B-Instruct instead of the provided annotations from Llama 3.1 70B. The dataset is a randomly sampled 467k document subset of the FineWeb dataset, which contains filtered documents crawled from the web. Please see https://arxiv.org/abs/2406.17557 for more details. <br>
111
 
112
  ### Evaluation Results
113
+ The models were shown to be useful in classifying high-quality content for LLM pretraining as part of an ensemble in the [Nemotron-CC](https://arxiv.org/abs/2412.02595) paper. See Table 9 from the paper below.
114
+
115
+ <img src="https://huggingface.co/nvidia/nemocurator-fineweb-mixtral-edu-classifier/resolve/main/table_9.PNG" alt="image" style="width:750px;">
116
+
117
+ In the table above, "Ours-mistral" refers to the NemoCurator FineWeb Mixtral Edu Classifier, and "Ours-nemotron-340B" refers to the NemoCurator FineWeb Nemotron-4 Edu Classifier (this model). "Ours-ensembled" incudes the NemoCurator FineWeb Mixtral Edu Classifier, NemoCurator FineWeb Nemotron-4 Edu Classifier, and DCLM.
118
 
119
  ## Inference
120
  - Engine: Python 3.10 and PyTorch