IrohXu

/

stable-diffusion-3-inpainting

like 6

You can check our plan here:
https://github.com/huggingface/diffusers/pull/8709

In current version you don't need the new model weight.
There is an alpha version for 33 channel's input: https://huggingface.co/George0667/SD3_inpaint_alpha

Hi, @IrohXu
For 33 channels transformer, do we need train a transformer from scratch or fully fine-tune (all parameters from transformer) the SD3's transformer?
Thanks!

Yes. I am training a version based on SD-3.5 now. Will see if it can work.

Thanks for the reply!
I am also curious about the dataset size and how the mask generated? Random generate or use SAM to segment?
@IrohXu

We are using SAM to segment the mask first, but it seems that the dataset for training an inpainting model should be very large.