metadata

datasets:
  - yuvalkirstain/pickapic_v2
language:
  - en
library_name: diffusers
pipeline_tag: text-to-image
license: openrail++

Diffusion Model Alignment Using Direct Preference Optimization

Direct Preference Optimization (DPO) for text-to-image diffusion models is a method to align diffusion models to text human preferences by directly optimizing on human comparison data. Please check our paper at Diffusion Model Alignment Using Direct Preference Optimization.

This model is fine-tuned from stable-diffusion-xl-base-1.0 on offline human preference data pickapic_v2.

Code

The code is available here.

SD1.5

We also have a model finedtuned from stable-diffusion-v1-5 available at dpo-sd1.5-text2image-v1.

A quick example

from diffusers import StableDiffusionXLPipeline, UNet2DConditionModel
import torch

# load pipeline
model_id = "stabilityai/stable-diffusion-xl-base-1.0"
pipe = StableDiffusionXLPipeline.from_pretrained(model_id, torch_dtype=torch.float16, variant="fp16", use_safetensors=True).to("cuda")

# load finetuned model
unet_id = "mhdang/dpo-sdxl-text2image-v1"
unet = UNet2DConditionModel.from_pretrained(unet_id, subfolder="unet", torch_dtype=torch.float16)
pipe.unet = unet
pipe = pipe.to("cuda")

prompt = "Two cats playing chess on a tree branch"
image = pipe(prompt, guidance_scale=5).images[0].resize((512,512))
    
image.save("cats_playing_chess.png")

More details coming soon.