Update content.py

content.py

@@ -7,8 +7,14 @@ HEADER_MARKDOWN = """
 Welcome to the leaderboard!  
 Here you can compare models on tasks in Czech language and/or submit your own model. We use our modified fork of [lm-evaluation-harness](https://github.com/DCGM/lm-evaluation-harness) to evaluate every model under same protocol.
 
+
 - Head to **Submission** page to learn about submission details.
-- See **About** page for brief description of our evaluation protocol & win score mechanism, citation information, and future directions for this benchmark.
+- See **About** page for brief description of our evaluation protocol & win score mechanism, citation information, and future directions for this benchmark. 
+- __How scoring works__:
+  - On each task, the __Duel Win Score__ reports proportion of won duels.
+  - Category scores are obtained by averaging across category tasks.
+  - __Average__ Duel Win Scores are an average over category scores.
+- All public submissions are shared in [CZLC/LLM_benchmark_data](https://huggingface.co/datasets/CZLC/LLM_benchmark_data) dataset.
 - In submission page, __you can obtain results on leaderboard without publishing them__.
     - First step is "pre-submission", and after this is done (significance tests can take up to an hour), the results can be submitted if you'd like to.
 
@@ -87,7 +93,6 @@ We use the following metrics for following tasks:
 - Fixed-class Classification: average area under the curve (one-vs-all average)
 - Multichoice Classification: accuracy
 - Question Answering: exact match
-- Summarization: rouge-raw (2-gram)
 - Language Modeling : word-level perplexity
 
 On every task, for every metric we compute test for statistical significance at α=0.05, i.e., the probability that performance model A is equal to the performance model B is estimated to be less then 0.05.
@@ -97,8 +102,12 @@ We use the following tests, with varying statistical power:
 - summarization & perplexity: bootstrapping.
 
 ### Duel Scoring Mechanism, Win Score
-On each task, each model is scored to each model (up to top-50 currently submitted models). For each model, record proportion of won duels: **Win Score**(WS).
-Next, the **Category Win Score**(CWS), is computed as an average over model's WSs in that category. Similarly, 🇨🇿 **BenCzechMark Win Score** is computed as model's average CWS across categories. 
+- We refer to each test of significance as **duel**. Model A _won_ the duel if it is sigificantly better than model B.
+- On each task, models are compared in duels with up to the top-50 currently submitted models.
+- For each model, the **Win Score** (WS) is calculated as the proportion of duels won by that model.
+- The **Category Win Score** (CWS) is the average of a model's WS across all tasks in a specific category.
+- The 🇨🇿 **BenCzechMark Win Score** is the overall score for a model, computed as the average of its CWS across all categories.
+
 The properties of this ranking mechanism include:
 - Ranking can change after every submission.
 - The across-task aggregation is interpretable: in words, it measures the average proportion of times the model is better.
@@ -108,14 +117,14 @@ The properties of this ranking mechanism include:
 The models submitted to leaderboard by the authors were evaluated in following setup:
 - max input length: 2048 tokens
 - number of shown examples (few-shot mechanism): 3-shot
-- truncation: smart truncation
+- truncation: smart truncation (few-shot samples are truncated before the task description)
 - log-probability aggregation: average-pooling
 - chat templates: not used
 
 ## Citation
 You can use the following citation for this leaderboard and our upcoming work.
 ```bibtex
-@article{fajcik2024benczechmark,
+@article{2024benczechmark,
   title = {{B}en{C}zech{M}ark: A Czech-centric Multitask and Multimetric Benchmark for Language Models with Duel Scoring Mechanism},
   author = {Martin Fajcik and Martin Docekal and Jan Dolezal and Karel Ondrej and Karel Benes and Jan Kapsa and Michal Hradis and Zuzana Neverilova and Ales Horak and Michal Stefanik and Adam Jirkovsky and David Adamczyk and Jan Hula and Jan Sedivy and Hynek Kydlicek},
   year = {2024},