Update t2i-adapter_diffusers_xl_ related information
Browse files
README.md
CHANGED
|
@@ -1,3 +1,43 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-sa-4.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-sa-4.0
|
| 3 |
---
|
| 4 |
+
|
| 5 |
+
# About This Repository
|
| 6 |
+
|
| 7 |
+
This repository has been established to house the controlnet models that have been corrected by the [Diffus](https://www.diffus.graviti.com/?utm_source=huggingface&utm_campaign=sd_control_collection_fixes) team. These models are now compatible with the [sd-webui-controlnet](https://github.com/Mikubill/sd-webui-controlnet) extension.
|
| 8 |
+
|
| 9 |
+
# About Diffus
|
| 10 |
+
|
| 11 |
+
[Diffus](https://www.diffus.graviti.com/?utm_source=huggingface&utm_campaign=sd_control_collection_fixes) provides an online Stable Diffusion webui service. The core UI is a heavily modified version of the original [AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui). All modifications have been open-sourced in accordance with the original GPL license.
|
| 12 |
+
|
| 13 |
+
# About the Fixes
|
| 14 |
+
|
| 15 |
+
In the process of integrating controlnet with our platform, we identified issues in some widely used models that prevented them from working seamlessly with the sd-webui-controlnet extension. We diagnosed these issues, made necessary modifications to the models, and now wish to share our solutions with the broader community. That's the purpose of this repository.
|
| 16 |
+
|
| 17 |
+
## 1. Issues with t2i-adapter_diffusers_xl_ Series Models
|
| 18 |
+
|
| 19 |
+
The original models can be found at [lllyasviel/sd_control_collection](https://huggingface.co/lllyasviel/sd_control_collection/tree/main).
|
| 20 |
+
|
| 21 |
+
A common error encountered with these models is:
|
| 22 |
+
|
| 23 |
+
```python
|
| 24 |
+
RuntimeError: Error(s) in loading state_dict for Adapter:
|
| 25 |
+
size mismatch for body.2.in_conv.weight: copying a param with shape torch.Size([1280, 640, 1, 1]) from checkpoint, the shape in current model is torch.Size([640, 320, 1, 1]).
|
| 26 |
+
size mismatch for body.2.in_conv.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([640]).
|
| 27 |
+
```
|
| 28 |
+
This error arises due to a mismatch between the keys in the trained models and those defined for the controlnet xl t2i-adapter. This mismatch renders the following models incompatible with sd-webui-controlnet (v1.1.4):
|
| 29 |
+
|
| 30 |
+
- t2i-adapter_diffusers_xl_canny.safetensors
|
| 31 |
+
- t2i-adapter_diffusers_xl_depth_midas.safetensors
|
| 32 |
+
- t2i-adapter_diffusers_xl_depth_zoe.safetensors
|
| 33 |
+
- t2i-adapter_diffusers_xl_lineart.safetensors
|
| 34 |
+
- t2i-adapter_diffusers_xl_openpose.safetensors
|
| 35 |
+
- t2i-adapter_diffusers_xl_sketch.safetensors
|
| 36 |
+
|
| 37 |
+
We have corrected the keys in these models and repackaged them as safetensors. You can download these updated models for use in your local installation of the Automatic1111 stable diffusion webui. They should be placed in the following location:
|
| 38 |
+
|
| 39 |
+
```bash
|
| 40 |
+
stable-diffusion-webui/extensions/sd-webui-controlnet/models
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
Alternatively, you can use these models directly via our [online service](https://www.diffus.graviti.com/?utm_source=huggingface&utm_campaign=sd_control_collection_fixes), eliminating the need to maintain your own instance.
|