---
license: apache-2.0
datasets:
- ai4colonoscopy/ColonINST-v1
language:
- en
metrics:
- accuracy
base_model:
- microsoft/phi-1_5
library_name: adapter-transformers
pipeline_tag: image-text-to-text
tags:
- medical
- colonoscopy
- polyp
---

# ColonGPT (A colonoscopy-specific multimodal Language Model)
<p align="center">
    <img src="./assert/web_ui_stg1.gif" width="666px"/> <br />
    <em>The Gradio Web UI allows you to use our examples or upload your images for inference.</em>
</p>

📖 [Paper](https://arxiv.org/abs/2410.17241) | 🏠 [Home](https://github.com/ai4colonoscopy/IntelliScope) 

> This is the weight of the pre-alignment stage of ColonGPT-v1. 


Our ColonGPT is a standard multimodal language model, which contains four basic components: a language tokenizer, an visual encoder (🤗 [SigLIP-SO](https://huggingface.co/google/siglip-so400m-patch14-384)), a multimodal connector, and a language model (🤗 [Phi1.5](https://huggingface.co/microsoft/phi-1_5)). In this huggingface page, we provide a quick start for convenient of new users. For further details about ColonGPT, we highly recommend visiting our [homepage](https://github.com/ai4colonoscopy/IntelliScope). There, you'll find comprehensive usage instructions for our model and the latest advancements in intelligent colonoscopy technology.