File size: 2,183 Bytes
92a36f5 6f94dc1 92a36f5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
---
license: mit
---
# SLaVA-CXR: Small Language and Vision Assistant for Chest X-ray Report Automation
**SLaVA-CXR: Small Language and Vision Assistant for Chest X-ray Report Automation** [[Paper](https://arxiv.org/abs/2409.13321)] [[Code](https://github.com/knowlab/SLaVA-CXR)] [[Model](https://huggingface.co/bluesky333/SLaVA-CXR)] <br>
## Contents
- [Install](#install)
- [LLaVA-Phi Weights](#llava-weights)
- [Train](#train)
- [Evaluation](#evaluation)
## Environment
```Shell
conda create -n slava_cxr python=3.10 -y
conda activate slava_cxr
pip install --upgrade pip # enable PEP 660 support
pip install -e .
```
## Train
The training codes is made available. The training datasets are currently not available.
## Evaluation
Evaluation dataset can be any chest X-ray frontal view image paired with a report.
We used MIMIC-CXR and IU-Xray datasets in our paper for the evaluation.
We have included IU-Xray questions for impression and findings section automation.
Please download IU-Xray dataset [LINK](https://drive.google.com/file/d/1c0BXEuDy8Cmm2jfN0YYGkQxFZd2ZIoLg/view).
### Findings Generation
```Shell
CUDA_VISIBLE_DEVICES=0 python -m llava_phi.eval.model_vqa_slava_cxr \
--model-path ./SLaVA-CXR \
--question-file iuxray_sample_findings.jsonl \
--image-folder path_to_iuxray_images \
--answers-file findings_result.jsonl \
--conv-mode default \
--max_new_tokens 512
```
### Impression Summarization
```Shell
CUDA_VISIBLE_DEVICES=0 python -m llava_phi.eval.model_vqa_slava_cxr \
--model-path ./SLaVA-CXR \
--question-file iuxray_sample_impression.jsonl \
--image-folder path_to_iuxray_images \
--answers-file impression_result.jsonl \
--conv-mode default \
--max_new_tokens 256
```
## Citation
```bibtex
@article{wu2024slava,
title={SLaVA-CXR: Small Language and Vision Assistant for Chest X-ray Report Automation},
author={Wu, Jinge and Kim, Yunsoo and Shi, Daqian and Cliffton, David and Liu, Fenglin and Wu, Honghan},
journal={arXiv preprint arXiv:2409.13321},
year={2024}
}
```
## Acknowledgement
We used the LLaVA-Phi codes to train our model
- [LLaVA-Phi](https://github.com/zhuyiche/llava-phi)
|