Commit
•
8c0d393
1
Parent(s):
786faaa
Update README.md
Browse files
README.md
CHANGED
@@ -1,7 +1,6 @@
|
|
1 |
---
|
2 |
base_model: stabilityai/stable-diffusion-3.5-large
|
3 |
library_name: diffusers
|
4 |
-
license: other
|
5 |
tags:
|
6 |
- text-to-image
|
7 |
- diffusers-training
|
@@ -10,70 +9,77 @@ tags:
|
|
10 |
- sd3
|
11 |
- sd3-diffusers
|
12 |
- template:sd-lora
|
13 |
-
instance_prompt:
|
14 |
-
|
15 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
---
|
17 |
|
18 |
-
|
19 |
-
should probably proofread and complete it, then remove this comment. -->
|
20 |
-
|
21 |
-
|
22 |
-
# SD3 DreamBooth LoRA - alvarobartt/ghibli-characters-sd3.5-lora
|
23 |
|
24 |
<Gallery />
|
25 |
|
26 |
## Model description
|
27 |
|
28 |
-
|
29 |
-
|
30 |
-
The weights were trained using [DreamBooth](https://dreambooth.github.io/) with the [SD3 diffusers trainer](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_sd3.md).
|
31 |
|
32 |
-
|
33 |
|
34 |
-
|
35 |
|
36 |
-
|
37 |
|
38 |
-
##
|
39 |
-
|
40 |
-
[Download the *.safetensors LoRA](alvarobartt/ghibli-characters-sd3.5-lora/tree/main) in the Files & versions tab.
|
41 |
-
|
42 |
-
## Use it with the [🧨 diffusers library](https://github.com/huggingface/diffusers)
|
43 |
-
|
44 |
-
```py
|
45 |
-
from diffusers import AutoPipelineForText2Image
|
46 |
-
import torch
|
47 |
-
pipeline = AutoPipelineForText2Image.from_pretrained('stabilityai/stable-diffusion-3-medium-diffusers', torch_dtype=torch.float16).to('cuda')
|
48 |
-
pipeline.load_lora_weights('alvarobartt/ghibli-characters-sd3.5-lora', weight_name='pytorch_lora_weights.safetensors')
|
49 |
-
image = pipeline('Ghibli style human with pig face, staring while being mouth full of food, in a restaurant, warm light on night, sweating a lot').images[0]
|
50 |
-
```
|
51 |
|
52 |
-
|
53 |
-
|
54 |
-
- **LoRA**: download **[`diffusers_lora_weights.safetensors` here 💾](/alvarobartt/ghibli-characters-sd3.5-lora/blob/main/diffusers_lora_weights.safetensors)**.
|
55 |
-
- Rename it and place it on your `models/Lora` folder.
|
56 |
-
- On AUTOMATIC1111, load the LoRA by adding `<lora:your_new_name:1>` to your prompt. On ComfyUI just [load it as a regular LoRA](https://comfyanonymous.github.io/ComfyUI_examples/lora/).
|
57 |
-
|
58 |
-
For more details, including weighting, merging and fusing LoRAs, check the [documentation on loading LoRAs in diffusers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters)
|
59 |
-
|
60 |
-
## License
|
61 |
-
|
62 |
-
Please adhere to the licensing terms as described [here](https://huggingface.co/stabilityai/stable-diffusion-3-medium/blob/main/LICENSE).
|
63 |
-
|
64 |
-
|
65 |
-
## Intended uses & limitations
|
66 |
-
|
67 |
-
#### How to use
|
68 |
|
69 |
```python
|
70 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
71 |
```
|
72 |
|
73 |
-
|
74 |
-
|
75 |
-
[TODO: provide examples of latent issues and potential remediations]
|
76 |
|
77 |
-
|
78 |
|
79 |
-
|
|
|
|
1 |
---
|
2 |
base_model: stabilityai/stable-diffusion-3.5-large
|
3 |
library_name: diffusers
|
|
|
4 |
tags:
|
5 |
- text-to-image
|
6 |
- diffusers-training
|
|
|
9 |
- sd3
|
10 |
- sd3-diffusers
|
11 |
- template:sd-lora
|
12 |
+
instance_prompt: >-
|
13 |
+
Ghibli style [character description] with [distinctive features], [action or
|
14 |
+
pose], [environment or background], [lighting or atmosphere], [additional
|
15 |
+
details]
|
16 |
+
license: other
|
17 |
+
license_name: stabilityai-ai-community
|
18 |
+
license_link: https://huggingface.co/stabilityai/stable-diffusion-3.5-large/blob/main/LICENSE.md
|
19 |
+
widget:
|
20 |
+
- text: >-
|
21 |
+
Ghibli style futuristic stormtrooper with glossy white armor and a sleek
|
22 |
+
helmet, standing heroically on a lush alien planet, vibrant flowers blooming
|
23 |
+
around, soft sunlight illuminating the scene, a gentle breeze rustling the
|
24 |
+
leaves
|
25 |
+
output:
|
26 |
+
url: images/stormtrooper.jpg
|
27 |
+
datasets:
|
28 |
+
- alvarobartt/ghibli-characters
|
29 |
+
pipeline_tag: text-to-image
|
30 |
---
|
31 |
|
32 |
+
# Studio Ghibli Characters - FLUX.1 Dev LoRA
|
|
|
|
|
|
|
|
|
33 |
|
34 |
<Gallery />
|
35 |
|
36 |
## Model description
|
37 |
|
38 |
+
This repository contains the LoRA adapter for [StableDiffusion 3.5 Large](https://huggingface.co/stabilityai/stable-diffusion-3.5-large), fine-tuned using [Diffusers + Dreambooth](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/train_dreambooth_lora_sd3.py)
|
39 |
+
with Studio Ghibli images free-to-use downloaded manually from https://ghibli.jp and hosted at [alvarobartt/ghibli-characters](https://huggingface.co/datasets/alvarobartt/ghibli-characters).
|
|
|
40 |
|
41 |
+
## Prompt Template
|
42 |
|
43 |
+
You should use the following template (defined when annotating the images with the captions) to trigger the image generation:
|
44 |
|
45 |
+
"Ghibli style [character description] with [distinctive features], [action or pose], [environment or background], [lighting or atmosphere], [additional details]"
|
46 |
|
47 |
+
## Inference with `diffusers`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
48 |
|
49 |
+
> [!NOTE]
|
50 |
+
> Requires `diffusers` 0.31.0 or higher, see the [release notes](https://github.com/huggingface/diffusers/releases/tag/v0.31.0).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
51 |
|
52 |
```python
|
53 |
+
import torch
|
54 |
+
from diffusers import DiffusionPipeline
|
55 |
+
|
56 |
+
model_id = "stabilityai/stable-diffusion-3.5-large"
|
57 |
+
adapter_id = "alvarobartt/ghibli-characters-sd3.5-lora"
|
58 |
+
|
59 |
+
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16)
|
60 |
+
pipeline.load_lora_weights(adapter_id)
|
61 |
+
pipeline.to("cuda")
|
62 |
+
|
63 |
+
prompt = (
|
64 |
+
"Ghibli style futuristic stormtrooper with glossy white armor and a sleek helmet,"
|
65 |
+
" standing heroically on a lush alien planet, vibrant flowers blooming around, soft"
|
66 |
+
" sunlight illuminating the scene, a gentle breeze rustling the leaves"
|
67 |
+
)
|
68 |
+
|
69 |
+
image = pipeline(
|
70 |
+
prompt=prompt,
|
71 |
+
num_inference_steps=30,
|
72 |
+
width=1024,
|
73 |
+
height=768,
|
74 |
+
guidance_scale=3.5,
|
75 |
+
).images[0]
|
76 |
+
|
77 |
+
image.save("ghibli.png", format="PNG")
|
78 |
```
|
79 |
|
80 |
+
## Disclaimer
|
|
|
|
|
81 |
|
82 |
+
This fine-tune is for **personal-use only**, with **no-commercial purposes** as stated within the licensing.
|
83 |
|
84 |
+
The StableDiffusion 3.5 Large fine-tunes fall under the same license as StableDiffusion 3.5 Large i.e. https://huggingface.co/stabilityai/stable-diffusion-3.5-large/blob/main/LICENSE.md;
|
85 |
+
and the Studio Ghibli dataset is released with a custom non-commercial license based on the interpretation of the findings on their website.
|