|
--- |
|
license: mit |
|
--- |
|
|
|
# Model Card for Model longluu/Clinical-NER-MedMentions-GatorTronS |
|
The model is an NER LLM algorithm that can classify each word in a text into different clinical categories. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
The base pretrained model is GatorTronS which was trained on billions of words in various clinical texts (https://huggingface.co/UFNLP/gatortronS). |
|
Then using the MedMentions dataset (https://arxiv.org/pdf/1902.09476v1.pdf), I fine-tuned the model for NER task in which the model can classify each word in a text into different clinical categories. |
|
The category system is a simplified version of UMLS concept system and consists of 21 categories: |
|
"['Living Beings', 'Virus']", "['Living Beings', 'Bacterium']", "['Anatomy', 'Anatomical Structure']", "['Anatomy', 'Body System']", "['Anatomy', 'Body Substance']", "['Disorders', 'Finding']", "['Disorders', 'Injury or Poisoning']", "['Phenomena', 'Biologic Function']", "['Procedures', 'Health Care Activity']", "['Procedures', 'Research Activity']", "['Devices', 'Medical Device']", "['Concepts & Ideas', 'Spatial Concept']", "['Occupations', 'Biomedical Occupation or Discipline']", "['Organizations', 'Organization']", "['Living Beings', 'Professional or Occupational Group']", "['Living Beings', 'Population Group']", "['Chemicals & Drugs', 'Chemical']", "['Objects', 'Food']", "['Concepts & Ideas', 'Intellectual Product']", "['Physiology', 'Clinical Attribute']", "['Living Beings', 'Eukaryote']", 'None' |
|
|
|
### Model Sources [optional] |
|
The github code associated with the model can be found here: https://github.com/longluu/LLM-NER-clinical-text. |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
The MedMentions dataset contain 4,392 abstracts released in PubMed®1 between January 2016 and January 2017. The abstracts were manually annotated for biomedical concepts. Details are provided in https://arxiv.org/pdf/1902.09476v1.pdf and data is in https://github.com/chanzuckerberg/MedMentions. |
|
|
|
#### Training Hyperparameters |
|
|
|
The hyperparameters are --batch_size 6 |
|
--num_train_epochs 6 |
|
--learning_rate 5e-5 |
|
--weight_decay 0.01 |
|
|
|
## Evaluation |
|
### Testing Data, Factors & Metrics |
|
#### Testing Data |
|
The model was trained and validated on train and validation sets. Then it was tested on a separate test set. |
|
Note that some concepts in the test set were not available in the train and validatin sets. |
|
|
|
#### Metrics |
|
Here we use several metrics for classification tasks including macro-average F1, precision, recall and Matthew correlation. |
|
|
|
### Results |
|
{'f1': 0.63101050849362, |
|
'precision': 0.6714479559316552, |
|
'recall': 0.6145681950682534, |
|
'matthews_correlation': 0.7203455851532575} |
|
|
|
## Model Card Contact |
|
Feel free to reach out to me at [email protected] if you have any question or suggestion. |
