OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting

ICCV 2025

Model Description

OmniPaint is a unified framework for object-oriented image editing that disentangles object insertion and removal into two specialized inpainting tasks. Built on top of FLUX.1-dev, OmniPaint uses LoRA adapters to enable:

• Object Removal — Remove foreground objects and their visual effects (shadows, reflections) using only object masks
• Object Insertion — Seamlessly insert objects into existing scenes with harmonious lighting and perspective

The model also introduces a novel CFD (Content-aware Frechet Distance) metric for reference-free evaluation of object removal quality.

Model Files

File	Description
weights/omnipaint_remove.safetensors	LoRA weights for object removal
weights/omnipaint_insert.safetensors	LoRA weights for object insertion
embeddings/remove.npz	Pre-computed text embeddings for removal
embeddings/insert.npz	Pre-computed text embeddings for insertion

Usage

Installation

git clone https://github.com/yeates/OmniPaint.git
cd OmniPaint
bash scripts/setup.sh

This will install dependencies and automatically download model weights from this repository.

Object Removal

python scripts/omnipaint_remove.py \
  --input ./demo_assets/removal_samples/images/5.jpg \
  --mask ./demo_assets/removal_samples/masks/5.png \
  --output_dir ./outputs \
  --seed 42 --steps 28 --device cuda:0

Object Insertion

python scripts/omnipaint_insert.py \
  --background ./demo_assets/insertion_samples/backgrounds/background-2.png \
  --mask ./demo_assets/insertion_samples/masks/mask-2.png \
  --subject ./demo_assets/insertion_samples/subjects/subject-2.png \
  --output_dir ./outputs \
  --seed 42 --steps 28 --device cuda:0

Gradio Demo

bash scripts/app_setup.sh  # one-time setup (~15 min)
python app.py

Technical Details

• Base model: FLUX.1-dev (FluxTransformer2DModel)
• Training method: LoRA fine-tuning with disentangled adapters for insertion and removal
• Framework: Diffusers + PEFT
• Inference steps: 28 (default)
• Guidance scale: 3.5 (default)

Citation

@inproceedings{yu2025omnipaint,
  title={Omnipaint: Mastering object-oriented editing via disentangled insertion-removal inpainting},
  author={Yu, Yongsheng and Zeng, Ziyun and Zheng, Haitian and Luo, Jiebo},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={17324--17334},
  year={2025}
}

License

This model is released under the MIT License.