Yoruba Sentiment Analysis with CNN and Afriberta

This repository contains a PyTorch model for sentiment analysis of Yoruba text. The model utilizes a Convolutional Neural Network (CNN) architecture on top of a pre-trained Afriberta model, specifically "Davlan/naija-twitter-sentiment-afriberta-large".

Model Description

The model consists of the following components:

Afriberta Base: The pre-trained Afriberta model serves as a powerful feature extractor for Yoruba text.
CNN Layers: Multiple 1D convolutional layers with varying kernel sizes capture local patterns and n-gram features from the Afriberta embeddings.
Max Pooling: Max pooling layers extract the most salient features from the convolutional outputs.
Dropout: Dropout regularization helps prevent overfitting.
Fully Connected Layer: A final fully connected layer maps the concatenated pooled features to sentiment classes.

Intended Uses & Limitations

This model is designed for sentiment analysis of Yoruba text and can be applied to various use cases, such as:

Social Media Monitoring: Analyze sentiment expressed in Yoruba tweets or social media posts.
Customer Feedback Analysis: Understand customer sentiment towards products or services in Yoruba.
Opinion Mining: Extract opinions and sentiments from Yoruba text data.

Limitations:

The model's performance may be limited by the size and quality of the training data.
It may not generalize well to domains significantly different from the training data.
As with any language model, there's a risk of bias and potential for misuse.

Training and Evaluation Data

The model was trained on a dataset of Yoruba tweets annotated with sentiment labels. The dataset was split into training, validation, and test sets.

Training Procedure

The model was trained using the following steps:

Data Preprocessing: Text data was tokenized using the Afriberta tokenizer.
Model Initialization: The SentimentCNNModel was initialized with the pre-trained Afriberta model and CNN layers.
Optimization: The model was trained using the Adam optimizer and cross-entropy loss.
Early Stopping: Training was stopped early based on validation loss to prevent overfitting.

Evaluation Results

The model achieved the following performance on the test set:

Test Loss: [0.6707]
F1-Score: [0.8095]

How to Use

Install Dependencies: Ensure you have PyTorch and Transformers installed:
```
pip install torch transformers
```

Load the Model: You can load the model using the Hugging Face transformers library:

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_name = "Testys/cnn_sent_yor"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

Make Predictions: Use the tokenizer to prepare your input text and the model to get predictions:

def predict(text):
    inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=128)
    outputs = model(**inputs)
    return torch.softmax(outputs.logits, dim=1).items()

sample_text = "Your Yoruba text here"
prediction = predict(sample_text)
print("Sentiment:", prediction)

Citing the Model

If you use this model in your research, please cite it using the following format:

@misc{your_model_name,
  author = {Your Name},
  title = {Yoruba Sentiment Analysis with CNN and Afriberta},
  year = {2024},
  publisher = {Hugging Face's Model Hub},
  journal = {Hugging Face's Model Hub},
  howpublished = {\\url{https://huggingface.co/your_model_name}}
}

License

This model is open-sourced under the MIT license. The license allows commercial use, modification, distribution, and private use.

Contact Information

For any queries regarding the model, feel free to reach out via GitHub or direct email:

GitHub: [https://github.com/dev-tyta]
Email: [[email protected]]

Testys
/

cnn_sent_yor