Update README.md

README.md

@@ -925,22 +925,153 @@ Here, we present results for seven categories of tasks in Spanish, Catalan, Basq
 
 Further details on all tasks and criteria, a full list of results compared to other baselines, a discussion of the model's performance across tasks and its implications, and details regarding problem-solving with task implementation will soon be available in the technical report.
 
-| **Category** | **Dataset** | **Metric** | **es** | **ca** | **gl** | **eu** | **en** |
-|---------|---------|-----------|-------|-------|-------|-------|-------|
-| **Commonsense Reasoning** | **XStoryCloze** | Ending Coherence (1 to 5) | 2.36/0.66 | 2.49/0.76 | 2.45/0.68 | 2.30/0.67 | 3.06/0.77 |
-| **Paraphrasing** | **PAWS** | Paraphrase Completeness (0/1) | 0.60/0.15 | 0.54/0.17 | 0.64/0.14 | ----/---- | 0.79/0.11 |
-| | | Paraphrase Generation (1 to 5) | 2.89/1.46 | 2.71/1.70 | 2.80/1.21 | ----/---- | 3.64/0.80 |
-| | | Paraphrase Grammatical Correctness (0/1) | 0.74/0.13 | 0.68/0.15 | 0.78/0.10 | ----/---- | 0.89/0.07 |
-| **Reading Comprehension** | **Belebele** | Passage Comprehension (1 to 5) | 3.05/0.60 | 2.81/0.66 | 2.74/0.78 | 2.52/0.46 | 3.11/0.71 |
-| | | Answer Relevance (0/1) | 0.74/0.09 | 0.66/0.11 | 0.65/0.12 | 0.59/0.12 | 0.75/0.09 |
-| **Extreme Summarization** | **XLSum & caBreu & summarization_gl** | Extreme Summarization Informativeness (1 to 5) | 3.07/0.39 | 3.33/0.43 | 3.11/0.36 | ----/---- | 3.06/0.35 |
-| | | Extreme Summarization Conciseness (1 to 5) | 2.92/0.42 | 2.67/0.54 | 2.93/0.39 | ----/---- | 3.13/0.31 |
-| **Mathematics** | **mgsm** | Reasoning Capability (1 to 5) | 1.89/0.47 | 1.91/0.45 | 1.97/0.43 | 2.17/0.44 | 2.16/0.56 |
-| | | Mathematical Correctness (0/1) | 0.24/0.10 | 0.28/0.11 | 0.27/0.11 | 0.44/0.13 | 0.27/0.10 |
-| **Translation form Language** | **FLoRes** | Translation Fluency (1 to 5) | 3.74/0.15 | 3.69/0.22 | ----/---- | ----/---- | 3.69/0.18 |
-| | | Translation Accuracy (1 to 5) | 4.01/0.24 | 3.98/0.31 | ----/---- | ----/---- | 3.98/0.25 |
-| **Translation to Language** | **FLoRes** | Translation Fluency (1 to 5) | 3.75/0.14 | 3.69/0.17 | ----/---- | ----/---- | 4.09/0.16 |
-| | | Translation Accuracy (1 to 5) | 4.08/0.22 | 3.98/0.24 | ----/---- | ----/---- | 4.47/0.18 |
+<style type="text/css">
+.tg {border-collapse:collapse;border-spacing:0;}
+.tg td{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;
+ overflow:hidden;padding:10px 5px;word-break:normal;}
+.tg th{border-color:black;border-style:solid;border-width:1px;font-family:Arial, sans-serif;font-size:14px;
+ font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;}
+.tg .tg-0pky{border-color:inherit;text-align:left;vertical-align:top}
+</style>
+<table class="tg"><thead>
+ <tr>
+ <th class="tg-0pky"><span style="font-weight:bold">Category</span></th>
+ <th class="tg-0pky"><span style="font-weight:bold">Dataset</span></th>
+ <th class="tg-0pky"><span style="font-weight:bold">Criteria</span></th>
+ <th class="tg-0pky"><span style="font-weight:bold">es</span></th>
+ <th class="tg-0pky"><span style="font-weight:bold">ca</span></th>
+ <th class="tg-0pky"><span style="font-weight:bold">gl</span></th>
+ <th class="tg-0pky"><span style="font-weight:bold">eu</span></th>
+ <th class="tg-0pky"><span style="font-weight:bold">en</span></th>
+ </tr></thead>
+<tbody>
+ <tr>
+ <td class="tg-0pky">Commonsense Reasoning</td>
+ <td class="tg-0pky">XStoryCloze</td>
+ <td class="tg-0pky">Ending coherence</td>
+ <td class="tg-0pky">2.36/0.66</td>
+ <td class="tg-0pky">2.49/0.76</td>
+ <td class="tg-0pky">2.45/0.68</td>
+ <td class="tg-0pky">2.30/0.67</td>
+ <td class="tg-0pky">3.06/0.77</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky" rowspan="3">Paraphrasing</td>
+ <td class="tg-0pky" rowspan="3">PAWS</td>
+ <td class="tg-0pky">Completeness `(B)`</td>
+ <td class="tg-0pky">0.60/0.15</td>
+ <td class="tg-0pky">0.54/0.17</td>
+ <td class="tg-0pky">0.64/0.14</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">0.79/0.11</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky">Paraphrase generation</td>
+ <td class="tg-0pky">2.89/1.46</td>
+ <td class="tg-0pky">2.71/1.70</td>
+ <td class="tg-0pky">2.80/1.21</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">3.64/0.80</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky">Grammatical correctness `(B)`</td>
+ <td class="tg-0pky">0.74/0.13</td>
+ <td class="tg-0pky">0.68/0.15</td>
+ <td class="tg-0pky">0.78/0.10</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">0.89/0.07</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky" rowspan="2">Reading Comprehension</td>
+ <td class="tg-0pky" rowspan="2">Belebele</td>
+ <td class="tg-0pky">Passage comprehension</td>
+ <td class="tg-0pky">3.05/0.60</td>
+ <td class="tg-0pky">2.81/0.66</td>
+ <td class="tg-0pky">2.74/0.78</td>
+ <td class="tg-0pky">2.52/0.46</td>
+ <td class="tg-0pky">3.11/0.71</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky">Answer relevance `(B)`</td>
+ <td class="tg-0pky">0.74/0.09</td>
+ <td class="tg-0pky">0.66/0.11</td>
+ <td class="tg-0pky">0.65/0.12</td>
+ <td class="tg-0pky">0.59/0.12</td>
+ <td class="tg-0pky">0.75/0.09</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky" rowspan="2">Extreme Summarization</td>
+ <td class="tg-0pky" rowspan="2">XLSum &amp; caBreu &amp; summarization_gl</td>
+ <td class="tg-0pky">Informativeness</td>
+ <td class="tg-0pky">3.07/0.39</td>
+ <td class="tg-0pky">3.33/0.43</td>
+ <td class="tg-0pky">3.11/0.36</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">3.06/0.35</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky">Conciseness</td>
+ <td class="tg-0pky">2.92/0.42</td>
+ <td class="tg-0pky">2.67/0.54</td>
+ <td class="tg-0pky">2.93/0.39</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">3.13/0.31</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky" rowspan="2">Math</td>
+ <td class="tg-0pky" rowspan="2">MGSM</td>
+ <td class="tg-0pky">Reasoning capability</td>
+ <td class="tg-0pky">1.89/0.47</td>
+ <td class="tg-0pky">1.91/0.45</td>
+ <td class="tg-0pky">1.97/0.43</td>
+ <td class="tg-0pky">2.17/0.44</td>
+ <td class="tg-0pky">2.16/0.56</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky">Mathematical correctness `(B)`</td>
+ <td class="tg-0pky">0.24/0.10</td>
+ <td class="tg-0pky">0.28/0.11</td>
+ <td class="tg-0pky">0.27/0.11</td>
+ <td class="tg-0pky">0.44/0.13</td>
+ <td class="tg-0pky">0.27/0.10</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky" rowspan="2">Translation form Language</td>
+ <td class="tg-0pky" rowspan="2">FLORES-200</td>
+ <td class="tg-0pky">Fluency</td>
+ <td class="tg-0pky">3.74/0.15</td>
+ <td class="tg-0pky">3.69/0.22</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">3.69/0.18</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky">Accuracy</td>
+ <td class="tg-0pky">4.01/0.24</td>
+ <td class="tg-0pky">3.98/0.31</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">3.98/0.25</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky" rowspan="2">Translation to Language</td>
+ <td class="tg-0pky" rowspan="2">FLORES-200</td>
+ <td class="tg-0pky">Fluency</td>
+ <td class="tg-0pky">3.75/0.14</td>
+ <td class="tg-0pky">3.69/0.17</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">4.09/0.16</td>
+ </tr>
+ <tr>
+ <td class="tg-0pky">Accuracy</td>
+ <td class="tg-0pky">4.08/0.22</td>
+ <td class="tg-0pky">3.98/0.24</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">-- / --</td>
+ <td class="tg-0pky">4.47/0.18</td>
+ </tr>
+</tbody></table>
 
 ---