Update README.md

README.md

@@ -26,10 +26,10 @@ It has different versions that were trained for different variants of Portuguese
 namely the European variant from Portugal (PT-PT) and the American variant from Brazil (PT-BR), 
 and it is distributed free of charge and under a most permissible license.
 
-**Albertina PT-PT** is the version for **European Portuguese from Portugal**, 
+**Albertina PT-BR** is the version for **American Portuguese from Brazil**, 
 and to the best of our knowledge, at the time of its initial distribution,
-it was the first competitive encoder for this language and variant 
-that had been made publicly available and distributed for reuse.
+it set a new state of the art for this language and variant 
+for a model that had been made publicly available and distributed for reuse.
 
 It was developped by a joint team from the University of Lisbon and the University of Porto, Portugal. 
 For further details, check the respective publication:
@@ -55,16 +55,21 @@ Please use the above cannonical reference when using or citing this model.
 
 # Model Description
 
-**This model card is for Albertina-PT-PT**, with 900M parameters, 24 layers and a hidden size of 1536.
+**This model card is for Albertina-PT-BR**, with 900M parameters, 24 layers and a hidden size of 1536.
 
-This model is distributed free of charge under the [MIT](https://choosealicense.com/licenses/mit/) license (permits commercial use, distribution, modification and private use).
+This model is distributed free of charge with the license granted to the [BrWac](https://huggingface.co/datasets/brwac) data set on which it was trained, 
+a license for non-commercial use.
 
 
 <br>
 
 # Training Data
 
-**Albertina PT-PT** was trained over a data set that resulted from gathering some openly available corpora of European Portuguese from the following sources:
+
+**Albertina PT-BR** was trained over the [BrWac](https://huggingface.co/datasets/brwac) data set.
+
+
+**Albertina PT-PT**, in turn, was trained over a data set that resulted from gathering some openly available corpora of European Portuguese from the following sources:
 
 - [OSCAR](https://huggingface.co/datasets/oscar-corpus/OSCAR-2301): the OSCAR data set includes documents in more than one hundred languages, including Portuguese, and it is widely used in the literature. It is the result of a selection performed over the [Common Crawl](https://commoncrawl.org/) data set, crawled from the Web, that retains only pages whose metadata indicates permission to be crawled, that performs deduplication, and that removes some boilerplate, among other filters. Given that it does not discriminate between the Portuguese variants, we performed extra filtering by retaining only documents whose meta-data indicate the Internet country code top-level domain of Portugal. We used the January 2023 version of OSCAR, which is based on the November/December 2022 version of Common Crawl.
 - [DCEP](https://joint-research-centre.ec.europa.eu/language-technology-resources/dcep-digital-corpus-european-parliament_en): the Digital Corpus of the European Parliament is a multilingual corpus including documents in all official EU languages published on the European Parliament&#39;s official website. We retained its European Portuguese portion.
@@ -72,8 +77,6 @@ This model is distributed free of charge under the [MIT](https://choosealicense.
 - [ParlamentoPT](https://www.parlamento.pt/): the ParlamentoPT is a data set we obtained by gathering the publicly available documents with the transcription of the debates in the Portuguese Parliament.
 
 
-**Albertina PT-BR**, in turn, was trained over the [BrWac](https://huggingface.co/datasets/brwac) data set.
-
 
 ## Preprocessing
 
@@ -85,12 +88,6 @@ We skipped the default filtering of stopwords since it would disrupt the syntact
 
 As codebase, we resorted to the [DeBERTa V2 XLarge](https://huggingface.co/microsoft/deberta-v2-xlarge), for English.
 
-To train **Albertina-PT-PT**, the data set was tokenized with the original DeBERTa tokenizer with a 128 token sequence truncation and dynamic padding.
-The model was trained using the maximum available memory capacity resulting in a batch size of 832 samples (52 samples per GPU and applying gradient accumulation in order to approximate the batch size of the PT-BR model).
-Similarly to the PT-BR variant above, we opted for a learning rate of 1e-5 with linear decay and 10k warm-up steps.
-However, since the number of training examples is approximately twice of that in the PT-BR variant, we reduced the number of training epochs to half and completed only 25 epochs, which resulted in approximately 245k steps.
-The model was trained for 3 days on a2-highgpu-8gb Google Cloud A2 VMs with 8 GPUs, 96 vCPUs and 680 GB of RAM.
-
 To train **Albertina-PT-BR** the BrWac data set was tokenized with the original DeBERTA tokenizer with a 128 token sequence truncation and dynamic padding.
 The model was trained using the maximum available memory capacity resulting in a batch size of 896 samples (56 samples per GPU without gradient accumulation steps).
 We chose a learning rate of 1e-5 with linear decay and 10k warm-up steps based on the results of exploratory experiments. 
@@ -98,6 +95,13 @@ In total, around 200k training steps were taken across 50 epochs.
 The model was trained for 1 day and 11 hours on a2-megagpu-16gb Google Cloud A2 VMs with 16 GPUs, 96 vCPUs and 1.360 GB of RAM.
 
 
+To train **Albertina-PT-PT**, the data set was tokenized with the original DeBERTa tokenizer with a 128 token sequence truncation and dynamic padding.
+The model was trained using the maximum available memory capacity resulting in a batch size of 832 samples (52 samples per GPU and applying gradient accumulation in order to approximate the batch size of the PT-BR model).
+Similarly to the PT-BR variant above, we opted for a learning rate of 1e-5 with linear decay and 10k warm-up steps.
+However, since the number of training examples is approximately twice of that in the PT-BR variant, we reduced the number of training epochs to half and completed only 25 epochs, which resulted in approximately 245k steps.
+The model was trained for 3 days on a2-highgpu-8gb Google Cloud A2 VMs with 8 GPUs, 96 vCPUs and 680 GB of RAM.
+
+
 <br>
 
 # Evaluation