|
--- |
|
base_model: stabilityai/stable-diffusion-3.5-large |
|
library_name: diffusers |
|
tags: |
|
- text-to-image |
|
- diffusers-training |
|
- diffusers |
|
- lora |
|
- sd3 |
|
- sd3-diffusers |
|
- template:sd-lora |
|
instance_prompt: >- |
|
Ghibli style [character description] with [distinctive features], [action or |
|
pose], [environment or background], [lighting or atmosphere], [additional |
|
details] |
|
license: other |
|
license_name: stabilityai-ai-community |
|
license_link: https://huggingface.co/stabilityai/stable-diffusion-3.5-large/blob/main/LICENSE.md |
|
widget: |
|
- text: >- |
|
Ghibli style futuristic stormtrooper with glossy white armor and a sleek |
|
helmet, standing heroically on a lush alien planet, vibrant flowers blooming |
|
around, soft sunlight illuminating the scene, a gentle breeze rustling the |
|
leaves |
|
output: |
|
url: images/stormtrooper.jpg |
|
datasets: |
|
- alvarobartt/ghibli-characters |
|
pipeline_tag: text-to-image |
|
--- |
|
|
|
# Studio Ghibli Characters - SD 3.5 Large LoRA |
|
|
|
<Gallery /> |
|
|
|
## Model description |
|
|
|
This repository contains the LoRA adapter for [StableDiffusion 3.5 Large](https://huggingface.co/stabilityai/stable-diffusion-3.5-large), fine-tuned using [Diffusers + Dreambooth](https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/train_dreambooth_lora_sd3.py) |
|
with Studio Ghibli images free-to-use downloaded manually from https://ghibli.jp and hosted at [alvarobartt/ghibli-characters](https://huggingface.co/datasets/alvarobartt/ghibli-characters). |
|
|
|
## Prompt Template |
|
|
|
You should use the following template (defined when annotating the images with the captions) to trigger the image generation: |
|
|
|
"Ghibli style [character description] with [distinctive features], [action or pose], [environment or background], [lighting or atmosphere], [additional details]" |
|
|
|
## Inference with `diffusers` |
|
|
|
> [!NOTE] |
|
> Requires `diffusers` 0.31.0 or higher, see the [release notes](https://github.com/huggingface/diffusers/releases/tag/v0.31.0). |
|
|
|
```python |
|
import torch |
|
from diffusers import DiffusionPipeline |
|
|
|
model_id = "stabilityai/stable-diffusion-3.5-large" |
|
adapter_id = "alvarobartt/ghibli-characters-sd3.5-lora" |
|
|
|
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) |
|
pipeline.load_lora_weights(adapter_id) |
|
pipeline.to("cuda") |
|
|
|
prompt = ( |
|
"Ghibli style futuristic stormtrooper with glossy white armor and a sleek helmet," |
|
" standing heroically on a lush alien planet, vibrant flowers blooming around, soft" |
|
" sunlight illuminating the scene, a gentle breeze rustling the leaves" |
|
) |
|
|
|
image = pipeline( |
|
prompt=prompt, |
|
num_inference_steps=30, |
|
width=1024, |
|
height=768, |
|
guidance_scale=3.5, |
|
).images[0] |
|
|
|
image.save("ghibli.png", format="PNG") |
|
``` |
|
|
|
## Disclaimer |
|
|
|
This fine-tune is for **personal-use only**, with **no-commercial purposes** as stated within the licensing. |
|
|
|
The StableDiffusion 3.5 Large fine-tunes fall under the same license as StableDiffusion 3.5 Large i.e. https://huggingface.co/stabilityai/stable-diffusion-3.5-large/blob/main/LICENSE.md; |
|
and the Studio Ghibli dataset is released with a custom non-commercial license based on the interpretation of the findings on their website. |
