PORTULAN
/

albertina-1b5-portuguese-ptpt-encoder

albertina-100m-portuguese-ptpt

albertina-100m-portuguese-ptbr

albertina-900m-portuguese-ptpt

albertina-900m-portuguese-ptbr

albertina-1b5-portuguese-ptpt

albertina-1b5-portuguese-ptbr

foundation model

Inference Endpoints

Model card Files Files and versions Community

jarodrigues commited on Feb 26, 2024

Commit

65d4c84

·

verified ·

1 Parent(s): 5e4d2d1

Update README.md

Files changed (1) hide show

README.md +10 -8

README.md CHANGED Viewed

@@ -116,17 +116,19 @@ We opted for a learning rate of 1e-5 with linear decay and 10k warm-up steps.
 The base model version was evaluated on downstream tasks, namely the translations into PT-PT of the English data sets used for a few of the tasks in the widely-used [GLUE benchmark](https://huggingface.co/datasets/glue).
-## GLUE tasks translated
-We resorted to [GLUE-PT](https://huggingface.co/datasets/PORTULAN/glue-ptpt), a **PT-PT version of the GLUE** benchmark.
-We automatically translated the same four tasks from GLUE using [DeepL Translate](https://www.deepl.com/), which specifically provides translation from English to PT-PT as an option.
-| Model                     | RTE (Accuracy) | WNLI (Accuracy)| MRPC (F1) | STS-B (Pearson) |
-|---------------------------|----------------|----------------|-----------|-----------------|
-| **Albertina 1.5B PT-PT**  | **0.?**        | 0.             | **0.    **| **0.    **      |
-| **Albertina PT-PT(900M)** |  0.8339        | 0.4225         | 0.9171    | 0.8801          |
-| **Albertina 100M PT-PT**  |  0.5848        | 0.5634         | 0.8793    | 0.8624          |
 <br>

 The base model version was evaluated on downstream tasks, namely the translations into PT-PT of the English data sets used for a few of the tasks in the widely-used [GLUE benchmark](https://huggingface.co/datasets/glue).
+## GLUE and SUPERGLUE tasks translated
+We resorted to [HyperGlue-PT](?), a **PT-PT version of the GLUE and SUPERGLUE** benchmark.
+We automatically translated the tasks from GLUE and SUPERGLUE using [DeepL Translate](https://www.deepl.com/), which specifically provides translation from English to PT-PT as an option.
+| Model                     | RTE (Accuracy) | WNLI (Accuracy)| MRPC (F1) | STS-B (Pearson) | COPA (Accuracy) | CB (F1) | MultiRC (F1) | BoolQ (Accuracy)
+|---------------------------|----------------|----------------|-----------|-----------------|-----------------|---------|--------------|
+| **Albertina 1.5B PT-PT**  | **0.?**        | 0.             | **0.    **| **0.    **      | ?               |         |              |
+| **Albertina PT-PT(900M)** |  0.8339        | 0.4225         | 0.9171    | 0.8801          | 0.7300          | 0.4739  | 0.6782       | 0.8437
+| **Albertina 100M PT-PT**  |  0.5848        | 0.5634         | 0.8793    | 0.8624          | n.a.            | 0.4734  | 0.6564       | 0.7700
 <br>