In-BoXBART
An instruction-based unified model for performing various biomedical tasks.
You may want to check out
- Our paper (NAACL 2022 Findings): In-BoXBART: Get Instructions into Biomedical Multi-Task Learning
- GitHub: Click Here
This work explores the impact of instructional prompts on biomedical Multi-Task Learning. We introduce the BoX, a collection of 32 instruction tasks for Biomedical NLP across (X) various categories. Using this meta-dataset, we propose a unified model termed In-BoXBART, that can jointly learn all tasks of the BoX without any task-specific modules. To the best of our knowledge, this is the first attempt to propose a unified model in the biomedical domain and use instructions to achieve generalization across several biomedical tasks.
Direct Use
It is possible to use this model to generate text, which is useful for experimentation and understanding its capabilities. It should not be directly used for production or work that may directly impact people.
How to Use
You can very easily load the models with Transformers, instead of downloading them manually. The BART-base model is the backbone of our model. Here is how to use the model in PyTorch:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("cogint/in-boxbart")
model = AutoModelForSeq2SeqLM.from_pretrained("cogint/in-boxbart")
Or just clone the model repo
git lfs install
git clone https://huggingface.co/cogint/in-boxbart
Inference Example
Here, we provide an example for the "Document Classification" (HoC dataset) task. Once you load the model from huggigface for inference, you can append instruction given in ./templates
for that particular dataset with input instance. Below is an example of one instance.
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("cogint/in-boxbart")
model = AutoModelForSeq2SeqLM.from_pretrained("cogint/in-boxbart")
# Input shows how we have appended instruction from our file for HoC dataset with instance.
input = "Instruction: Definition: In this task, you are given a medical text related to cancer. Your job is to classify into zero or more classes from (1) Sustaining proliferative signaling, (2) Resisting cell death, (3) Genomic instability and mutation, (4) Activating invasion and metastasis, (5) Tumor promoting inflammation, (6) Evading growth suppressors, (7) Inducing angiogenesis (8) Enabling replicative immortality, (9) Avoiding immune destruction and (10) Cellular energetics., Positive Examples: [[input: Studies of cell-cycle progression showed that the anti-proliferative effect of Fan was associated with an increase in the G1/S phase of PC3 cells ., output: Evading growth suppressors, Sustaining proliferative signaling, explanation: Given text is classified into two categories, hence, generated label is 'Evading growth suppressors, Sustaining proliferative signaling'.] ]; Instance: input: Similar to previous studies utilizing IGF-1 , pretreatment with Roscovitine leads to a significant up-regulation of p21 expression and a significant decrease in the number of PCNA positive cells ., output: ?"
tokenized_input= tokenizer(input)
# Ideal output for this input is 'Sustaining proliferative signaling'
output = model(tokenized_input)
BibTeX Entry and Citation Info
If you are using our model, please cite our paper:
@inproceedings{parmar-etal-2022-boxbart,
title = "In-{B}o{XBART}: Get Instructions into Biomedical Multi-Task Learning",
author = "Parmar, Mihir and
Mishra, Swaroop and
Purohit, Mirali and
Luo, Man and
Mohammad, Murad and
Baral, Chitta",
booktitle = "Findings of the Association for Computational Linguistics: NAACL 2022",
month = jul,
year = "2022",
address = "Seattle, United States",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-naacl.10",
doi = "10.18653/v1/2022.findings-naacl.10",
pages = "112--128",
}
- Downloads last month
- 23