license: apache-2.0
datasets:
- Zakia/drugscom_reviews
language:
- en
metrics:
- accuracy
library_name: transformers
pipeline_tag: text-classification
tags:
- health
- medicine
- patient reviews
- drug reviews
- depression
- text classification
widget:
- text: >-
This medication has changed my life for the better. I've experienced no
side effects and my symptoms of depression have significantly decreased.
example_title: Example 1
- text: >-
I've had a terrible experience with this medication. It made me feel
nauseous and I didn't notice any improvement in my condition.
example_title: Example 2
Model Card for Zakia/distilbert-drugscom_depression_reviews
This model is a DistilBERT-based classifier fine-tuned on drug reviews for the depression medical condition from Drugs.com. The dataset used for fine-tuning is the Zakia/drugscom_reviews dataset, which is filtered for the condition 'Depression'. The base model for fine-tuning was the distilbert-base-uncased.
Model Details
Model Description
- Developed by: Zakia
- Model type: Text Classification
- Language(s) (NLP): English
- License: Apache 2.0
- Finetuned from model: distilbert-base-uncased
Uses
Direct Use
This model is intended to classify drug reviews into high or low quality, aiding in the analysis of patient feedback on depression medications.
Out-of-Scope Use
This model is not designed to diagnose or treat depression or to replace professional medical advice.
Bias, Risks, and Limitations
The model may inherit biases present in the dataset and should not be used as the sole decision-maker for healthcare or treatment options.
Recommendations
Use the model as a tool to support, not replace, professional judgment.
How to Get Started with the Model
Use the code below to get started with the model.
from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch.nn.functional as F
model_name = "Zakia/distilbert-drugscom_depression_reviews"
model = AutoModelForSequenceClassification.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)
# Define a function to print predictions with labels
def print_predictions(review_text, model, tokenizer):
inputs = tokenizer(review_text, return_tensors="pt")
outputs = model(**inputs)
predictions = F.softmax(outputs.logits, dim=-1)
# LABEL_0 is for low quality and LABEL_1 for high quality
print(f"Review: \"{review_text}\"")
print(f"Prediction: {{'LABEL_0 (Low quality)': {predictions[0][0].item():.4f}, 'LABEL_1 (High quality)': {predictions[0][1].item():.4f}}}\n")
# High quality review example
high_quality_review = "This medication has changed my life for the better. I've experienced no side effects and my symptoms of depression have significantly decreased."
print_predictions(high_quality_review, model, tokenizer)
# Low quality review example
low_quality_review = "I've had a terrible experience with this medication. It made me feel nauseous and I didn't notice any improvement in my condition."
print_predictions(low_quality_review, model, tokenizer)
Training Details
Training Data
The model was fine-tuned on a dataset of drug reviews specifically related to depression, filtered from Drugs.com. This dataset is accessible from Zakia/drugscom_reviews on Hugging Face datasets (condition = 'Depression') for 'train'. Number of records in train dataset: 9069 rows.
Training Procedure
Preprocessing
The reviews were cleaned and preprocessed to remove quotes, HTML tags and decode HTML entities. A new column called 'high_quality_review' was also added to the reviews. 'high_quality_review' was computed as 1 if rating > 5 (positive rating) and usefulCount > the 75th percentile of usefulCount (65) or 0, otherwise. Train dataset high_quality_review counts: Counter({0: 6949, 1: 2120}) Then: This training data was balanced by downsampling low quality reviews (high_quality_review = 0). The final training data had 4240 rows of reviews: Train dataset high_quality_review counts: Counter({0: 2120, 1: 2120})
Training Hyperparameters
- Learning Rate: 3e-5
- Batch Size: 16
- Epochs: 1
Evaluation
Testing Data, Factors & Metrics
Testing Data
The model was tested on a dataset of drug reviews specifically related to depression, filtered from Drugs.com. This dataset is accessible from Zakia/drugscom_reviews on Hugging Face datasets (condition = 'Depression') for 'test'. Number of records in test dataset: 3095 rows.
Preprocessing
The reviews were cleaned and preprocessed to remove quotes, HTML tags and decode HTML entities. A new column called 'high_quality_review' was also added to the reviews. 'high_quality_review' was computed as 1 if rating > 5 (positive rating) and usefulCount > the 75th percentile of usefulCount (65) or 0, otherwise. Note: the 75th percentile of usefulCount is based on the train dataset. Test dataset high_quality_review counts: Counter({0: 2365, 1: 730})
Metrics
The model's performance was evaluated based on accuracy.
Results
The fine-tuning process yielded the following results:
| Epoch | Training Loss | Validation Loss | Accuracy |
|---|---|---|---|
| 1 | 0.38 | 0.80 | 0.77 |
The model demonstrates its capability to classify drug reviews as high or low quality with an accuracy of 77%. Low Quality: high_quality_review=0 High Quality: high_quality_review=1
Technical Specifications
Model Architecture and Objective
DistilBERT model architecture was used, with a binary classification head for high and low quality review classification.
Compute Infrastructure
The model was trained using a T4 GPU on Google Colab.
Hardware
T4 GPU via Google Colab.
Citation
If you use this model, please cite the original DistilBERT paper:
BibTeX:
@article{sanh2019distilbert,
title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
author={Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},
journal={arXiv preprint arXiv:1910.01108},
year={2019}
}
APA:
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108.
Glossary
- Low Quality Review: high_quality_review=0
- High Quality Review: high_quality_review=1
More Information
For further queries or issues with the model, please use the discussions section on this model's Hugging Face page.
Model Card Authors
- Zakia
Model Card Contact
For more information or inquiries regarding this model, please use the discussions section on this model's Hugging Face page.