metadata

license: mit
language:
  - en
tags:
  - medical
  - vision
widget:
  - src: https://d168r5mdg5gtkq.cloudfront.net/medpix/img/full/synpic9078.jpg
    candidate_labels: Chest X-Ray, Brain MRI, Abdomen CT Scan
    example_title: Abdomen CT Scan

Model Card for PubMedCLIP

PubMedCLIP is a fine-tuned version of CLIP for the medical domain trained on a large number of medical image–text pairs obtained from PubMed articles.

Model Details

Model Description

PubMedCLIP was trained on the Radiology Objects in COntext (ROCO) dataset, a large-scale multimodal medical imaging dataset. The ROCO dataset includes diverse imaging modalities (such as ultrasound, X-Ray, MRI, etc.) from various human body regions (such as head, neck, spine, etc.) captured from open-access PubMed articles. The texts used for training PubMedCLIP were taken from the short captions associated with the images in the dataset.

The authors of PubMedCLIP have released three different pre-trained models at this link using ResNet-50, ResNet-50x4 and ViT32 as image encoders. This repository includes only the ViT32 variant.

Model Sources

Repository: Official GitHub Repository
Paper [optional]: {{ paper | default("[More Information Needed]", true)}}
Demo [optional]: {{ demo | default("[More Information Needed]", true)}}

Uses

Direct Use

Downstream Use [optional]

Out-of-Scope Use

{{ out_of_scope_use | default("[More Information Needed]", true)}}

Bias, Risks, and Limitations

{{ bias_risks_limitations | default("[More Information Needed]", true)}}

Recommendations

{{ bias_recommendations | default("Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.", true)}}

How to Get Started with the Model

Use the code below to get started with the model.

{{ get_started_code | default("[More Information Needed]", true)}}

Training Details

Training Data

Training Procedure

Preprocessing [optional]

Training Hyperparameters

Training regime: {{ training_regime | default("[More Information Needed]", true)}}

Speeds, Sizes, Times [optional]

{{ speeds_sizes_times | default("[More Information Needed]", true)}}

Evaluation

Testing Data, Factors & Metrics

Testing Data

Factors

{{ testing_factors | default("[More Information Needed]", true)}}

Metrics

{{ testing_metrics | default("[More Information Needed]", true)}}

Results

Summary

Model Examination [optional]

{{ model_examination | default("[More Information Needed]", true)}}

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: {{ hardware | default("[More Information Needed]", true)}}
Hours used: {{ hours_used | default("[More Information Needed]", true)}}
Cloud Provider: {{ cloud_provider | default("[More Information Needed]", true)}}
Compute Region: {{ cloud_region | default("[More Information Needed]", true)}}
Carbon Emitted: {{ co2_emitted | default("[More Information Needed]", true)}}

Technical Specifications [optional]

Model Architecture and Objective

Compute Infrastructure

{{ compute_infrastructure | default("[More Information Needed]", true)}}

Hardware

Software

Citation [optional]

BibTeX:

{{ citation_bibtex | default("[More Information Needed]", true)}}

APA:

Glossary [optional]

More Information [optional]

{{ more_information | default("[More Information Needed]", true)}}

Model Card Authors [optional]

{{ model_card_authors | default("[More Information Needed]", true)}}

Model Card Contact

{{ model_card_contact | default("[More Information Needed]", true)}}