---
datasets:
- jigsaw_unintended_bias
language:
- en
---

# Model Name

Toxicity Classifier with Debiaser

## Model description

This model is a text classification model trained on a large dataset of comments to predict whether a given comment contains biased language or not. 
The model is based on DistilBERT architecture and fine-tuned on a labeled dataset of toxic and non-toxic comments. 

## Intended Use

This model is intended to be used to automatically detect  biased language in user-generated comments in various online platforms. 
It can also be used as a component in a larger pipeline for text classification, sentiment analysis, or bias detection tasks.

`````
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("shainaraza/toxity_classify_debiaser")

model = AutoModelForSequenceClassification.from_pretrained("shainaraza/toxity_classify_debiaser")

# Test the model with a sample comment
comment = "you are a dumb person."
inputs = tokenizer(comment, return_tensors="pt")
outputs = model(**inputs)
prediction = torch.argmax(outputs.logits, dim=1).item()

print(f"Comment: {comment}")
print(f"Prediction: {'biased' if prediction == 1 else 'not biased'}")

`````

## Training data

The model was trained on a labeled dataset of comments from various online platforms, which were annotated as toxic or non-toxic by human annotators. 

## Evaluation results

The model was evaluated on a separate test set of comments and achieved the following performance metrics:

- Accuracy: 0.95
- F1-score: 0.94
- ROC-AUC: 0.97

## Limitations and bias

This model has been trained and tested on comments from various online platforms, but its performance may be limited when applied to comments from different domains or languages.

## Conclusion

The Toxicity Classifier is a powerful tool for automatically detecting and flagging potentially biased language in user-generated comments. While there are some limitations to its performance and potential biases in the training data, the model's high accuracy and robustness make it a valuable asset for any online platform looking to improve the quality and inclusivity of its user-generated content.