Model Card

README.md

@@ -11,87 +11,97 @@ license: apache-2.0
 
 ### zho-eng
 
-* source group: Chinese 
-* target group: English 
-*  OPUS readme: [zho-eng](https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/zho-eng/README.md)
-
-*  model: transformer
-* source language(s): cjy_Hans cjy_Hant cmn cmn_Hans cmn_Hant gan lzh lzh_Hans nan wuu yue yue_Hans yue_Hant
-* target language(s): eng
-* model: transformer
-* pre-processing: normalization + SentencePiece (spm32k,spm32k)
-* download original weights: [opus-2020-07-17.zip](https://object.pouta.csc.fi/Tatoeba-MT-models/zho-eng/opus-2020-07-17.zip)
-* test set translations: [opus-2020-07-17.test.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/zho-eng/opus-2020-07-17.test.txt)
-* test set scores: [opus-2020-07-17.eval.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/zho-eng/opus-2020-07-17.eval.txt)
-
-## Benchmarks
-
-| testset               | BLEU  | chr-F |
-|-----------------------|-------|-------|
-| Tatoeba-test.zho.eng 	| 36.1 	| 0.548 |
-
+## Table of Contents
+- [Model Details](#model-details)
+- [Uses](#uses)
+- [Risks, Limitations and Biases](#risks-limitations-and-biases)
+- [Training](#training)
+- [Evaluation](#evaluation)
+- [Citation Information](#citation-information)
+- [How to Get Started With the Model](#how-to-get-started-with-the-model)
 
-### System Info: 
-- hf_name: zho-eng
+## Model Details
+- **Model Description:**
+- **Developed by:** Language Technology Research Group at the University of Helsinki
+- **Model Type:** Translation
+- **Language(s):**  
+  - Source Language:  Chinese
+  - Target Language: English
+- **License:** Apache-2.0
+- **Resources for more information:**
+  - [GitHub Repo](https://github.com/Helsinki-NLP/OPUS-MT-train)
 
-- source_languages: zho
 
-- target_languages: eng
+## Uses
 
-- opus_readme_url: https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/zho-eng/README.md
+#### Direct Use
 
-- original_repo: Tatoeba-Challenge
+This model can be used for translation and text-to-text generation.
 
-- tags: ['translation']
 
-- languages: ['zh', 'en']
+## Risks, Limitations and Biases
 
-- src_constituents: {'cmn_Hans', 'nan', 'nan_Hani', 'gan', 'yue', 'cmn_Kana', 'yue_Hani', 'wuu_Bopo', 'cmn_Latn', 'yue_Hira', 'cmn_Hani', 'cjy_Hans', 'cmn', 'lzh_Hang', 'lzh_Hira', 'cmn_Hant', 'lzh_Bopo', 'zho', 'zho_Hans', 'zho_Hant', 'lzh_Hani', 'yue_Hang', 'wuu', 'yue_Kana', 'wuu_Latn', 'yue_Bopo', 'cjy_Hant', 'yue_Hans', 'lzh', 'cmn_Hira', 'lzh_Yiii', 'lzh_Hans', 'cmn_Bopo', 'cmn_Hang', 'hak_Hani', 'cmn_Yiii', 'yue_Hant', 'lzh_Kana', 'wuu_Hani'}
+**CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.**
 
-- tgt_constituents: {'eng'}
+Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)).
 
-- src_multilingual: False
+Further details about the dataset for this model can be found in the OPUS readme: [zho-eng](https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/zho-eng/README.md)
 
-- tgt_multilingual: False
+## Training
 
-- prepro:  normalization + SentencePiece (spm32k,spm32k)
+#### System Information 
+* helsinki_git_sha: 480fcbe0ee1bf4774bcbe6226ad9f58e63f6c535
+* transformers_git_sha: 2207e5d8cb224e954a7cba69fa4ac2309e9ff30b
+* port_machine: brutasse
+* port_time: 2020-08-21-14:41
+* src_multilingual: False
+* tgt_multilingual: False
 
-- url_model: https://object.pouta.csc.fi/Tatoeba-MT-models/zho-eng/opus-2020-07-17.zip
-
-- url_test_set: https://object.pouta.csc.fi/Tatoeba-MT-models/zho-eng/opus-2020-07-17.test.txt
-
-- src_alpha3: zho
+#### Training Data
+##### Preprocessing
+* pre-processing: normalization + SentencePiece (spm32k,spm32k)
+* ref_len: 82826.0
+* dataset: [opus](https://github.com/Helsinki-NLP/Opus-MT)
+* download original weights: [opus-2020-07-17.zip](https://object.pouta.csc.fi/Tatoeba-MT-models/zho-eng/opus-2020-07-17.zip)
 
-- tgt_alpha3: eng
+* test set translations: [opus-2020-07-17.test.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/zho-eng/opus-2020-07-17.test.txt)
 
-- short_pair: zh-en
 
-- chrF2_score: 0.5479999999999999
+## Evaluation
 
-- bleu: 36.1
+#### Results
 
-- brevity_penalty: 0.948
+* test set scores: [opus-2020-07-17.eval.txt](https://object.pouta.csc.fi/Tatoeba-MT-models/zho-eng/opus-2020-07-17.eval.txt)
 
-- ref_len: 82826.0
+* brevity_penalty: 0.948
 
-- src_name: Chinese
 
-- tgt_name: English
+## Benchmarks
 
-- train_date: 2020-07-17
+| testset               | BLEU  | chr-F |
+|-----------------------|-------|-------|
+| Tatoeba-test.zho.eng 	| 36.1 	| 0.548 |
 
-- src_alpha2: zh
+## Citation Information
 
-- tgt_alpha2: en
+```bibtex
+@InProceedings{TiedemannThottingal:EAMT2020,
+  author = {J{\"o}rg Tiedemann and Santhosh Thottingal},
+  title = {{OPUS-MT} — {B}uilding open translation services for the {W}orld},
+  booktitle = {Proceedings of the 22nd Annual Conferenec of the European Association for Machine Translation (EAMT)},
+  year = {2020},
+  address = {Lisbon, Portugal}
+ }
+```
 
-- prefer_old: False
+## How to Get Started With the Model
 
-- long_pair: zho-eng
+```python
+from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
 
-- helsinki_git_sha: 480fcbe0ee1bf4774bcbe6226ad9f58e63f6c535
+tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-zh-en")
 
-- transformers_git_sha: 2207e5d8cb224e954a7cba69fa4ac2309e9ff30b
+model = AutoModelForSeq2SeqLM.from_pretrained("Helsinki-NLP/opus-mt-zh-en")
+```
 
-- port_machine: brutasse
 
-- port_time: 2020-08-21-14:41