Spaces:
Running
Running
| """ | |
| This file contains the text content for the leaderboard client. | |
| """ | |
| HEADER_MARKDOWN = """ | |
| # 🇨🇿 BenCzechMark [Beta Preview] | |
| Welcome to the leaderboard! Here you can compare models on tasks in Czech language and/or submit your own model. Head to submission page to learn about submission details. | |
| We use our modified fork of [lm-evaluation-harness](https://github.com/DCGM/lm-evaluation-harness) to evaluate every model under same protocol. | |
| See about page for brief description of our evaluation protocol & win score mechanism, citation information, and future directions for this benchmark. | |
| """ | |
| LEADERBOARD_TAB_TITLE_MARKDOWN = """ | |
| ## Leaderboard | |
| """ | |
| SUBMISSION_TAB_TITLE_MARKDOWN = """ | |
| ## How to submit | |
| 1. Head down to our modified fork of [lm-evaluation-harness](https://github.com/DCGM/lm-evaluation-harness). | |
| Follow the instructions and evaluate your model on all 🇨🇿 BenCzechMark tasks, while logging your lm harness outputs into designated folder. | |
| 2. Use our script <TODO: add script> for processing log files from your designated folder into single compact submission file that contains everything we need. | |
| 3. Upload your file, and fill the form below! | |
| ## Submission | |
| To submit your model, please fill in the form below. | |
| - *Team name:* The name of your team, as it will appear on the leaderboard | |
| - *Model name:* The name of your model | |
| - *Model type:* The type of your model (chat, pretrained, ensemble) | |
| - *Parameters (B):* The number of parameters of your model in billions (10⁹) | |
| - *Input length (# tokens):* The number of input tokens that led to the results | |
| - *Precision:* The precision with which the results were obtained | |
| - *Description:* Short description of your submission (optional) | |
| - *Link to model:* Link to the model's repository or documentation | |
| - *Upload your results:* Results json file to submit | |
| After filling in the form, click the **Pre-submit model** button. | |
| This will run a comparison of your model with the existing leaderboard models. | |
| After the tournament is complete, you will be able to submit your model to the leaderboard. | |
| """ | |
| RANKING_AFTER_SUBMISSION_MARKDOWN = """ | |
| This is how will ranking look like after your submission: | |
| """ | |
| SUBMISSION_DETAILS_MARKDOWN = """ | |
| Do you really want to submit a model? This action is irreversible. | |
| """ | |
| MORE_DETAILS_MARKDOWN = """ | |
| ## Model details: | |
| """ | |
| MODAL_SUBMIT_MARKDOWN = """ | |
| Are you sure you want to submit your model? | |
| """ | |