---
license: cc-by-4.0
datasets:
- dsfsi/vukuzenzele-monolingual
- nchlt
language:
- tn
library_name: transformers
pipeline_tag: fill-mask
tags:
- masked langauge model
- setswana
---
# Model Card for PuoBERTa


## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

- **Developed by:** Vukosi Marivate, Moseli Mots'Oehli, Valencia Wagner,Richard Lastrucci and Isheanesu Dzingirai
- **Model type:** RoBERTa Model
- **Language(s) (NLP):** Setswana
- **License:** CC BY 4.0

<!--  ### Model Sources [optional] -->

<!-- Provide the basic links for the model. -->

<!--- **Repository:** [More Information Needed] . -->
<!-- - **Paper [optional]:** [More Information Needed] . -->
<!-- - **Demo [optional]:** [More Information Needed] . -->

## Uses

Pre-trained masked language model for Setswana. Model can be fine-tuned for downstream NLP tasks for Setswana. 

### Downstream Use 
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

[More Information Needed]

<!-- ### Out-of-Scope Use

This section addresses misuse, malicious use, and uses that the model will not work well for.

[More Information Needed]-->

<!-- ## Bias, Risks, and Limitations

This section is meant to convey both technical and sociotechnical limitations.

[More Information Needed]  -->

<!-- ### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

<!-- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

<!--  ## How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

<!-- ## Training Details

<!--  ### Training Data

<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

<!--  [More Information Needed]

### Training Procedure 

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

<!--  #### Preprocessing [optional]

[More Information Needed]


#### Training Hyperparameters

- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

<!--  #### Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

<!--  [More Information Needed]

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

<!-- ### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Data Card if possible. 

[More Information Needed]

#### Factors -->

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

<!-- #### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

<!-- ### Results

[More Information Needed]

#### Summary


## Model Examination [optional]  -->

<!-- Relevant interpretability work for the model goes here -->


<!-- R### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

[More Information Needed]

#### Hardware

[More Information Needed]

#### Software

[More Information Needed]

<!--  ## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

<!-- **BibTeX:**

[More Information Needed]

**APA:**

[More Information Needed] -->



## Model Card Authors

Vukosi Marivate

## Model Card Contact

vukosi.marivate@cs.up.ac.za