how to diffusers (#3)

- how to diffusers (7c5d0f9b82bd92221f1bc43608ea1208822023f7)


Co-authored-by: Radamés Ajna <[email protected]>

README.md

@@ -17,6 +17,70 @@ pipeline_tag: image-to-image
 This repo holds the safetensors & diffusers versions of the QR code conditioned ControlNet for Stable Diffusion v2.1.
 The Stable Diffusion 2.1 version is marginally more effective, as it was developed to address my specific needs. However, a 1.5 version model was also trained on the same dataset for those who are using the older version.
 
+## How to use with diffusers
+
+```bash
+pip -q install diffusers transformers accelerate torch xformers
+```
+
+```python
+import torch
+from PIL import Image
+from diffusers import StableDiffusionControlNetImg2ImgPipeline, ControlNetModel, DDIMScheduler
+from diffusers.utils import load_image
+
+controlnet = ControlNetModel.from_pretrained("DionTimmer/controlnet_qrcode-control_v11p_sd21",
+                                             torch_dtype=torch.float16)
+
+pipe = StableDiffusionControlNetImg2ImgPipeline.from_pretrained(
+    "runwayml/stable-diffusion-v1-5",
+    controlnet=controlnet,
+    safety_checker=None,
+    torch_dtype=torch.float16
+)
+
+pipe.enable_xformers_memory_efficient_attention()
+pipe.scheduler = DDIMScheduler.from_config(pipe.scheduler.config)
+pipe.enable_model_cpu_offload()
+
+def resize_for_condition_image(input_image: Image, resolution: int):
+    input_image = input_image.convert("RGB")
+    W, H = input_image.size
+    k = float(resolution) / min(H, W)
+    H *= k
+    W *= k
+    H = int(round(H / 64.0)) * 64
+    W = int(round(W / 64.0)) * 64
+    img = input_image.resize((W, H), resample=Image.LANCZOS)
+    return img
+
+
+# play with guidance_scale, controlnet_conditioning_scale and strength to make a valid QR Code Image
+
+# qr code image
+source_image = load_image("https://s3.amazonaws.com/moonup/production/uploads/6064e095abd8d3692e3e2ed6/A_RqHaAM6YHBodPLwqtjn.png")
+# initial image, anything
+init_image = load_image("https://s3.amazonaws.com/moonup/production/uploads/noauth/KfMBABpOwIuNolv1pe3qX.jpeg")
+condition_image = resize_for_condition_image(source_image, 768)
+init_image = resize_for_condition_image(init_image, 768)
+generator = torch.manual_seed(123121231)
+image = pipe(prompt="a bilboard in NYC with a qrcode",
+             negative_prompt="ugly, disfigured, low quality, blurry, nsfw", 
+             image=init_image,
+             control_image=condition_image,
+             width=768,
+             height=768,
+             guidance_scale=20,
+             controlnet_conditioning_scale=1.5,
+             generator=generator,
+             strength=0.9, 
+             num_inference_steps=150,
+            )
+
+image.images[0]
+
+```
+
 ## Performance and Limitations
 
 These models perform quite well in most cases, but please note that they are not 100% accurate. In some instances, the QR code shape might not come through as expected. You can increase the ControlNet weight to emphasize the QR code shape. However, be cautious as this might negatively impact the style of your output.**To optimize for scanning, please generate your QR codes with correction mode 'H' (30%).**