|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- Zakia/drugscom_reviews |
|
language: |
|
- en |
|
metrics: |
|
- accuracy |
|
library_name: transformers |
|
pipeline_tag: text-classification |
|
tags: |
|
- health |
|
- medicine |
|
- patient reviews |
|
- drug reviews |
|
- depression |
|
- text classification |
|
--- |
|
|
|
# Model Card for Zakia/distilbert-drugscom_depression_reviews |
|
|
|
This model is a DistilBERT-based classifier fine-tuned on drug reviews for the depression medical condition from Drugs.com. |
|
The dataset used for fine-tuning is the [Zakia/drugscom_reviews](https://huggingface.co/datasets/Zakia/drugscom_reviews) dataset, which is filtered for the condition 'Depression'. |
|
The base model for fine-tuning was the [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased). |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
|
|
- **Developed by:*Zakia* |
|
- **Model type:*Text Classification* |
|
- **Language(s) (NLP):*English* |
|
- **License:*Apache 2.0* |
|
- **Finetuned from model:*distilbert-base-uncased* |
|
|
|
## Uses |
|
|
|
### Direct Use |
|
|
|
This model is intended to classify drug reviews into high or low quality, aiding in the analysis of patient feedback on depression medications. |
|
|
|
### Out-of-Scope Use |
|
|
|
This model is not designed to diagnose or treat depression or to replace professional medical advice. |
|
|
|
## Bias, Risks, and Limitations |
|
|
|
The model may inherit biases present in the dataset and should not be used as the sole decision-maker for healthcare or treatment options. |
|
|
|
### Recommendations |
|
|
|
Use the model as a tool to support, not replace, professional judgment. |
|
|
|
## How to Get Started with the Model |
|
|
|
Use the code below to get started with the model. |
|
|
|
```python |
|
from transformers import AutoModelForSequenceClassification, AutoTokenizer |
|
import torch.nn.functional as F |
|
|
|
model_name = "Zakia/distilbert-drugscom_depression_reviews" |
|
model = AutoModelForSequenceClassification.from_pretrained(model_name) |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
|
|
# Define a function to print predictions with labels |
|
def print_predictions(review_text, model, tokenizer): |
|
inputs = tokenizer(review_text, return_tensors="pt") |
|
outputs = model(**inputs) |
|
predictions = F.softmax(outputs.logits, dim=-1) |
|
# LABEL_0 is for low quality and LABEL_1 for high quality |
|
print(f"Review: \"{review_text}\"") |
|
print(f"Prediction: {{'LABEL_0 (Low quality)': {predictions[0][0].item():.4f}, 'LABEL_1 (High quality)': {predictions[0][1].item():.4f}}}\n") |
|
|
|
# High quality review example |
|
high_quality_review = "This medication has changed my life for the better. I've experienced no side effects and my symptoms of depression have significantly decreased." |
|
print_predictions(high_quality_review, model, tokenizer) |
|
|
|
# Low quality review example |
|
low_quality_review = "I've had a terrible experience with this medication. It made me feel nauseous and I didn't notice any improvement in my condition." |
|
print_predictions(low_quality_review, model, tokenizer) |
|
``` |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
The model was fine-tuned on a dataset of drug reviews specifically related to depression, filtered from Drugs.com. |
|
This dataset is accessible from [Zakia/drugscom_reviews](https://huggingface.co/datasets/Zakia/drugscom_reviews) on Hugging Face datasets (condition = 'Depression') for 'train'. |
|
Number of records in train dataset: 9069 rows. |
|
|
|
### Training Procedure |
|
|
|
#### Preprocessing |
|
|
|
The reviews were cleaned and preprocessed to remove quotes, HTML tags and decode HTML entities. |
|
A new column called 'high_quality_review' was also added to the reviews. |
|
'high_quality_review' was computed as 1 if rating > 5 (positive rating) and usefulCount > the 75th percentile of usefulCount (65) or 0, otherwise. |
|
Train dataset high_quality_review counts: Counter({0: 6949, 1: 2120}) |
|
Then: |
|
This training data was balanced by downsampling low quality reviews (high_quality_review = 0). |
|
The final training data had 4240 rows of reviews: |
|
Train dataset high_quality_review counts: Counter({0: 2120, 1: 2120}) |
|
|
|
#### Training Hyperparameters |
|
|
|
- **Learning Rate: *3e-5* |
|
- **Batch Size:*16* |
|
- **Epochs:*1* |
|
|
|
## Evaluation |
|
|
|
### Testing Data, Factors & Metrics |
|
|
|
#### Testing Data |
|
|
|
The model was tested on a dataset of drug reviews specifically related to depression, filtered from Drugs.com. |
|
This dataset is accessible from [Zakia/drugscom_reviews](https://huggingface.co/datasets/Zakia/drugscom_reviews) on Hugging Face datasets (condition = 'Depression') for 'test'. |
|
Number of records in test dataset: 3095 rows. |
|
|
|
#### Preprocessing |
|
|
|
The reviews were cleaned and preprocessed to remove quotes, HTML tags and decode HTML entities. |
|
A new column called 'high_quality_review' was also added to the reviews. |
|
'high_quality_review' was computed as 1 if rating > 5 (positive rating) and usefulCount > the 75th percentile of usefulCount (65) or 0, otherwise. |
|
Note: the 75th percentile of usefulCount is based on the train dataset. |
|
Test dataset high_quality_review counts: Counter({0: 2365, 1: 730}) |
|
|
|
#### Metrics |
|
|
|
The model's performance was evaluated based on accuracy. |
|
|
|
### Results |
|
|
|
The fine-tuning process yielded the following results: |
|
|
|
| Epoch | Training Loss | Validation Loss | Accuracy | |
|
|-------|---------------|-----------------|----------| |
|
| 1 | 0.38 | 0.80 | 0.77 | |
|
|
|
The model demonstrates its capability to classify drug reviews as high or low quality with an accuracy of 77%. |
|
Low Quality: high_quality_review=0 |
|
High Quality: high_quality_review=1 |
|
|
|
## Technical Specifications |
|
|
|
### Model Architecture and Objective |
|
|
|
DistilBERT model architecture was used, with a binary classification head for high and low quality review classification. |
|
|
|
### Compute Infrastructure |
|
|
|
The model was trained using a T4 GPU on Google Colab. |
|
|
|
#### Hardware |
|
|
|
T4 GPU via Google Colab. |
|
|
|
## Citation |
|
|
|
If you use this model, please cite the original DistilBERT paper: |
|
|
|
**BibTeX:** |
|
|
|
```bibtex |
|
@article{sanh2019distilbert, |
|
title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter}, |
|
author={Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas}, |
|
journal={arXiv preprint arXiv:1910.01108}, |
|
year={2019} |
|
} |
|
``` |
|
**APA:** |
|
|
|
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. |
|
|
|
## Glossary |
|
|
|
- **Low Quality Review: *high_quality_review=0* |
|
- **High Quality Review:*high_quality_review=1* |
|
|
|
## More Information |
|
|
|
For further queries or issues with the model, please use the [discussions section on this model's Hugging Face page](https://huggingface.co/Zakia/distilbert-drugscom_depression_reviews/discussions). |
|
|
|
|
|
## Model Card Authors |
|
|
|
- Zakia |
|
|
|
## Model Card Contact |
|
|
|
For more information or inquiries regarding this model, please use the [discussions section on this model's Hugging Face page](https://huggingface.co/Zakia/distilbert-drugscom_depression_reviews/discussions). |
