File size: 2,043 Bytes
d6f9980 aea7293 d6f9980 aea7293 e28ef7c d6f9980 aea7293 d6f9980 aea7293 d6f9980 aea7293 d6f9980 aea7293 1156752 d6f9980 aea7293 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 |
---
license: apache-2.0
base_model: bert-base-uncased
tags:
- 'biology '
- NLP
- text-classification
- drugs
- BERT
metrics:
- accuracy
- precision
- recall
- f1
model-index:
- name: bert-drug-review-to-condition
results: []
language:
- en
library_name: transformers
datasets:
- Zakia/drugscom_reviews
---
# bert-drug-review-to-condition
This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4308
- Accuracy: 0.9209
- Precision: 0.9061
- Recall: 0.9209
- F1: 0.9106
## Model description
Fine-tuning of Bert model with drug-related data for the purpose of text classification
## Intended uses & limitations
Personal project.
## Training and evaluation data
Kallumadi,Surya and Grer,Felix. (2018). Drug Reviews (Drugs.com). UCI Machine Learning Repository. https://doi.org/10.24432/C5SK5S.
## Training procedure
Multiclass classification
The model predicts the 'condition' feature from the 'review' feature, only the first 21 conditions are selected.
The 'review' feature is lowercased, we select only values with at least 16 characters.
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3.0
### Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy | Precision | Recall | F1 |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:---------:|:------:|:------:|
| No log | 1.0 | 113 | 1.1375 | 0.7747 | 0.7301 | 0.7747 | 0.7450 |
| No log | 2.0 | 226 | 0.5595 | 0.8854 | 0.8675 | 0.8854 | 0.8728 |
| No log | 3.0 | 339 | 0.4308 | 0.9209 | 0.9061 | 0.9209 | 0.9106 |
### Framework versions
- Transformers 4.40.0
- Pytorch 2.2.1+cu121
- Datasets 2.19.0
- Tokenizers 0.19.1 |