metadata

license: apache-2.0
base_model: stabilityai/stable-diffusion-xl-base-1.0
tags:
  - stable-diffusion-xl
  - stable-diffusion-xl-diffusers
  - text-to-image
  - diffusers
  - controlnet
inference: false
language:
  - en
pipeline_tag: text-to-image

EcomXL Inpaint ControlNet

EcomXL contains a series of text-to-image diffusion models optimized for e-commerce scenarios, developed based on Stable Diffusion XL.
For e-commerce scenarios, we trained Inpaint ControlNet to control diffusion models. Unlike the inpaint controlnets used for general scenarios, this model is fine-tuned with instance masks to prevent foreground outpainting.

Examples

`Foreground`	`Mask`	`w/o instance mask`	`w/ instance mask`

Using this ControlNet with a control weight of 0.5 may achieve better results.

Usage with Diffusers

from diffusers import ControlNetModel
import torch

controlnet = ControlNetModel.from_pretrained(
    "alimama-creative/EcomXL_controlnet_inpaint", torch_dtype=torch.float16, use_safetensors=True
)

Training details

In the first phase, the model was trained on 12M laion2B and internal source images with random masks for 20k steps. In the second phase, the model was trained on 3M e-commerce images with the instance mask for 20k steps.
Mixed precision: FP16
Learning rate: 1e-4
batch size: 2048
Noise offset: 0.05