kaushalya commited on
Commit
ac2a427
·
1 Parent(s): af144e3

Update readme

Browse files
Files changed (1) hide show
  1. README.md +11 -4
README.md CHANGED
@@ -14,11 +14,11 @@ You can try a Streamlit demo app that uses this model on [🤗 Spaces](https://h
14
  ![Streamlit app](./assets/streamlit_app.png)
15
 
16
  🤗 Hub Model card: https://huggingface.co/flax-community/medclip-roco
17
- ## Dataset
18
 
19
- Each image is accompanied by a text caption. The caption length varies from a few characters (a single word) to 2,000 characters. During preprocessing we remove all images that has a caption shorter than 10 characters.
20
  Training set: 57,780 images with their caption.
21
- Validation set: 7.200
22
  Test set: 7,650
23
 
24
  [ ] Give an example
@@ -26,7 +26,7 @@ Test set: 7,650
26
  ## Installation 💽
27
  This repo depends on the master branch of [Hugging Face - Transformers library](https://github.com/huggingface/transformers). First you need to clone the transformers repository and then install it locally (preferably inside a virtual environment) with `pip install -e ".[flax]"`.
28
 
29
- ## Model
30
  You can load the pretrained model from the Hugging Face Hub with
31
  ```
32
  from medclip.modeling_hybrid_clip import FlaxHybridCLIP
@@ -39,6 +39,13 @@ You can fine-tune a CLIP model implemented in Flax by simply running `sh run_med
39
  This is the validation loss curve we observed when we trained the model using the `run_medclip.sh` script.
40
  ![Validation loss](./assets/val_loss.png)
41
 
 
 
 
 
 
 
 
42
  ## TODO
43
  [ ] Evaluation on down-stream tasks
44
 
 
14
  ![Streamlit app](./assets/streamlit_app.png)
15
 
16
  🤗 Hub Model card: https://huggingface.co/flax-community/medclip-roco
17
+ ## Dataset 🧩
18
 
19
+ Each image is accompanied by a textual caption. The caption length varies from a few characters (a single word) to 2,000 characters (multiple sentences). During preprocessing we remove all images that has a caption shorter than 10 characters.
20
  Training set: 57,780 images with their caption.
21
+ Validation set: 7,200
22
  Test set: 7,650
23
 
24
  [ ] Give an example
 
26
  ## Installation 💽
27
  This repo depends on the master branch of [Hugging Face - Transformers library](https://github.com/huggingface/transformers). First you need to clone the transformers repository and then install it locally (preferably inside a virtual environment) with `pip install -e ".[flax]"`.
28
 
29
+ ## The Model ⚙️
30
  You can load the pretrained model from the Hugging Face Hub with
31
  ```
32
  from medclip.modeling_hybrid_clip import FlaxHybridCLIP
 
39
  This is the validation loss curve we observed when we trained the model using the `run_medclip.sh` script.
40
  ![Validation loss](./assets/val_loss.png)
41
 
42
+ ## Limitations 🚨
43
+ The current model is capable of identifying if a given radiology image is a PET scan or an ultrasound scan. However it fails at identifying a brain scan from a lung scan. ❗️This model **should not** be used in a medical setting without further evaluations❗️.
44
+
45
+ ## Acknowledgements
46
+ Huge thanks to the Hugging Face 🤗 team and Google JAX/Flax team for organizing the community week and letting us use cloud compute for 2 weeks. We specially thank [@patil-suraj](https://github.com/patil-suraj) & [@patrickvonplaten](https://github.com/patrickvonplaten) for the continued support on Slack and the detailed feedback.
47
+
48
+
49
  ## TODO
50
  [ ] Evaluation on down-stream tasks
51