README.md

---
license: other
language:
- tr
library_name: transformers
pipeline_tag: text2text-generation
---


# Model Card for TURNA

<!-- Provide a quick summary of what the model is/does. -->

TURNA is a Turkish language model based on the UL2 framework which is suitable for both understanding and generation tasks.

Evaluations across three generation and six understanding tasks in Turkish show that TURNA outperforms several multilingual models and competes with monolingual Turkish models in understanding tasks.

The model is shared with the public to be used solely for non-commercial academic research purposes.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

- **Developed by:** Bogazici University Computer Engineering Department TABILAB (special thanks to VNGRS-AI for sharing their tokenizer)
- **Funded by:** We thank the Google TPU Research Cloud program for providing us with credits to pretrain our model on TPU v3-8 machines.
<!-- - **Shared by [optional]:** [More Information Needed] -->
- **Model type:** Transformer-based encoder-decoder
- **Language(s) (NLP):** Turkish
- **License:** The model is shared with the public to be used solely for non-commercial academic research purposes.

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [Training code](https://github.com/boun-tabi-LMG/turna), [Finetuning library](https://github.com/boun-tabi-LMG/turkish-lm-tuner)
- **Paper:** [Arxiv paper](https://arxiv.org/abs/)

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

This model can be used for research purposes. You give some text and this model tries to predict the next words.

### Downstream Use

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

This model can be finetuned using [our library](https://github.com/boun-tabi-LMG/turkish-lm-tuner) to solve your own task involving Turkish language.

This model can be further trained for behaving more helpful, less harmful and better for dialog use cases.

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

Any commercial or malicious activity.

## Bias, Risks, and Limitations

We refer to the Flan-T5's [official model card](https://arxiv.org/pdf/2210.11416.pdf):

> Language models, including Flan-T5, can potentially be used for language generation in a harmful way, according to Rae et al. (2021). Flan-T5 should not be used directly in any application, without a prior assessment of safety and fairness concerns specific to the application.

### Ethical considerations and risks

> ... (ed. The model) is fine-tuned on a large corpus of text data that was not filtered for explicit content or assessed for existing biases. As a result the model itself is potentially vulnerable to generating equivalently inappropriate content or replicating inherent biases in the underlying data.

### Known Limitations

> ... (ed. The model) has not been tested in real world applications.

### Sensitive Use:

> ... (ed. The model) should not be applied for any unacceptable use cases, e.g., generation of abusive speech.


## How to Get Started with the Model

You can find the technical usage guidance at our library's Github [page](https://github.com/boun-tabi-LMG/turkish-lm-tuner).

## Training Details

Refer to the paper for more information.

## Evaluation

Refer to the paper for more information.

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** TPU v3-8
- **Hours used:** About 400 hours
- **Cloud Provider:** Google Cloud
- **Compute Region:** europe-west4-a
- **Carbon Emitted:** 64.52 kg CO2_2

## Technical Specifications

Refer to the paper for more information.

## Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

Coming soon!

**APA:**

Coming soon!

## Model Card Authors

Paper authors.

## Model Card Contact

Onur Güngör