---
base_model: stabilityai/stable-diffusion-xl-base-1.0
library_name: diffusers
license: openrail++
tags:
- text-to-image
- text-to-image
- diffusers-training
- diffusers
- lora
- template:sd-lora
- stable-diffusion-xl
- stable-diffusion-xl-diffusers
datasets:
- data-is-better-together/open-image-preferences-v1-binarized
language:
- en
pipeline_tag: text-to-image
widget:
- text: An hyperrealistic astronaut riding a green horse
  output:
    url: images/example_p8cnk74hq.png

---


# Low Rank Adapted Supervised Fine Tuned Stable Diffusion XL

## Comparison

| Prompt | SDXL | Fine Tuned |
| :--: | :--: | :--: |
| a boat in the canals of Venice, painted in gouache with soft, flowing brushstrokes and vibrant, translucent colors, capturing the serene reflection on the water under a misty ambiance, with rich textures and a dynamic perspective | ![image/png](https://cdn-uploads.huggingface.co/production/uploads/608aabf24955d2bfc3cd99c6/CtoZWxDmANYm7d95I3Fcp.png) | ![image/png](https://cdn-uploads.huggingface.co/production/uploads/608aabf24955d2bfc3cd99c6/hAxmaL-robradk1x_KqwQ.png) |
| Grainy shot of a robot cooking in the kitchen, with soft shadows and nostalgic film texture. | ![image/png](https://cdn-uploads.huggingface.co/production/uploads/608aabf24955d2bfc3cd99c6/LTQI1NdaEjJUgeDpqzv7k.png) | ![image/png](https://cdn-uploads.huggingface.co/production/uploads/608aabf24955d2bfc3cd99c6/vAjnMCW0nmbV0zHKT8oCJ.png) |

## Model description

These are ariG23498/open-image-preferences-v1-sdxl-lora LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0.

The weights were trained using [DreamBooth](https://github.com/ariG23498/diffusers/blob/aritra/sdxl-lora/examples/dreambooth/train_dreambooth_lora_sdxl.py) using the
[open-image-preferences-v1-binarized](https://huggingface.co/datasets/data-is-better-together/open-image-preferences-v1-binarized) dataset.


## Use with `diffusers`

```py
from diffusers import AutoPipelineForText2Image
import torch

pipeline = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.bfloat16
).to('cuda')
pipeline.load_lora_weights('ariG23498/open-image-preferences-v1-sdxl-lora', weight_name='pytorch_lora_weights.safetensors')
prompt = "ENTER PROMPT"
image = pipeline(prompt).images[0]
```

## Command to train the model

```shell
!accelerate launch examples/dreambooth/train_dreambooth_lora_sdxl.py \
    --pretrained_model_name_or_path "stabilityai/stable-diffusion-xl-base-1.0" \
    --dataset_name "data-is-better-together/open-image-preferences-v1-binarized" \
    --hub_model_id "ariG23498/open-image-preferences-v1-sdxl-lora" \
    --push_to_hub \
    --output_dir "open-image-preferences-v1-sdxl-lora" \
    --image_column "chosen" \
    --caption_column "prompt" \
    --mixed_precision="bf16" \
    --resolution=1024 \
    --train_batch_size=1 \
    --repeats=1 \
    --report_to="wandb"\
    --gradient_accumulation_steps=1 \
    --gradient_checkpointing \
    --learning_rate=1.0 \
    --text_encoder_lr=1.0 \
    --optimizer="prodigy"\
    --lr_scheduler="constant" \
    --lr_warmup_steps=0 \
    --rank=8 \
    --checkpointing_steps=2000 \
    --seed="0"
```