Update README.md
Browse files
README.md
CHANGED
@@ -9,7 +9,7 @@ Data, knowledge store and source code to reproduce the baseline experiments for
|
|
9 |
|
10 |
## NEWS:
|
11 |
- 19.04.2024: The submisstion page (with eval.ai) for the shared-task is alive, you can participate by submitting your predictions [here](https://eval.ai/web/challenges/challenge-page/2285/overview)!
|
12 |
-
- 15.07.2024: To facilitate human evaluation we now ask the submission files to include a `scraped_text
|
13 |
|
14 |
|
15 |
## Dataset
|
@@ -131,7 +131,7 @@ python -m src.prediction.evaluate_veracity
|
|
131 |
|
132 |
To facilitate human evaluation, the submission file should include the text of the evidence documents used, retrieved through the `url` field. If external knowledge is utilized, please provide the scraped text. If our provided knowledge store is used, this can be achieved by running the following code block (see [veracity_with_scraped_text.py](https://huggingface.co/chenxwh/AVeriTeC/blob/main/src/prediction/veracity_with_scraped_text.py) for adding the text to the previous prediction file. An example output for the dev set is [here](https://huggingface.co/chenxwh/AVeriTeC/blob/main/data_store/dev_veracity_prediction_for_submission.json).
|
133 |
```bash
|
134 |
-
python -m src.prediction.veracity_with_scraped_text --knowledge_store_dir <
|
135 |
```
|
136 |
|
137 |
Each line of the final submission file is a json object with the following information:
|
|
|
9 |
|
10 |
## NEWS:
|
11 |
- 19.04.2024: The submisstion page (with eval.ai) for the shared-task is alive, you can participate by submitting your predictions [here](https://eval.ai/web/challenges/challenge-page/2285/overview)!
|
12 |
+
- 15.07.2024: To facilitate human evaluation we now ask the submission files to include a `scraped_text` field in your submission file, have a look in [here](https://huggingface.co/chenxwh/AVeriTeC#format-for-submission-files) for more information!
|
13 |
|
14 |
|
15 |
## Dataset
|
|
|
131 |
|
132 |
To facilitate human evaluation, the submission file should include the text of the evidence documents used, retrieved through the `url` field. If external knowledge is utilized, please provide the scraped text. If our provided knowledge store is used, this can be achieved by running the following code block (see [veracity_with_scraped_text.py](https://huggingface.co/chenxwh/AVeriTeC/blob/main/src/prediction/veracity_with_scraped_text.py) for adding the text to the previous prediction file. An example output for the dev set is [here](https://huggingface.co/chenxwh/AVeriTeC/blob/main/data_store/dev_veracity_prediction_for_submission.json).
|
133 |
```bash
|
134 |
+
python -m src.prediction.veracity_with_scraped_text --knowledge_store_dir <directory_of_the_knowledge_store>
|
135 |
```
|
136 |
|
137 |
Each line of the final submission file is a json object with the following information:
|