Update README.md
Browse files
README.md
CHANGED
|
@@ -49,12 +49,12 @@ The evaluation was done **five-shot** with native prompts on a sample of 100-100
|
|
| 49 |
|
| 50 |
Following the implementation of IFEval in OpenLLM leaderboard, we also implement SEA-IFEval to provide a comparison of the ability of the model to follow specific constraints in English and in SEA languages.
|
| 51 |
|
| 52 |
-
For more details on Llama3.1 70B CPT SEA-LIONv3 base benchmark performance, please refer to the SEA-HELM leaderboard, https://leaderboard.sea-lion.ai/
|
| 53 |
-
|
| 54 |
**SEA-IFEval**
|
| 55 |
|
| 56 |
SEA-IFEval evaluates a model's ability to adhere to constraints provided in the prompt, for example beginning a response with a specific word/phrase or answering with a certain number of sections. Additionally, accuracy is normalised by the proportion of responses in the correct language (if the model performs the task correctly but responds in the wrong language, it is judged to have failed the task).
|
| 57 |
|
|
|
|
|
|
|
| 58 |
## Technical Specifications
|
| 59 |
### Infrastructure
|
| 60 |
Llama3.1 70B CPT SEA-LIONv3 was trained in two stages using [MosaicML Composer](https://github.com/mosaicml/composer) on the following hardware:
|
|
|
|
| 49 |
|
| 50 |
Following the implementation of IFEval in OpenLLM leaderboard, we also implement SEA-IFEval to provide a comparison of the ability of the model to follow specific constraints in English and in SEA languages.
|
| 51 |
|
|
|
|
|
|
|
| 52 |
**SEA-IFEval**
|
| 53 |
|
| 54 |
SEA-IFEval evaluates a model's ability to adhere to constraints provided in the prompt, for example beginning a response with a specific word/phrase or answering with a certain number of sections. Additionally, accuracy is normalised by the proportion of responses in the correct language (if the model performs the task correctly but responds in the wrong language, it is judged to have failed the task).
|
| 55 |
|
| 56 |
+
For more details on Llama3.1 70B CPT SEA-LIONv3 base benchmark performance, please refer to the SEA-HELM leaderboard, https://leaderboard.sea-lion.ai/.
|
| 57 |
+
|
| 58 |
## Technical Specifications
|
| 59 |
### Infrastructure
|
| 60 |
Llama3.1 70B CPT SEA-LIONv3 was trained in two stages using [MosaicML Composer](https://github.com/mosaicml/composer) on the following hardware:
|