Spaces:

Duplicated from  John6666/sdxl-to-diffusers-v2

John6666

/

sdxl-to-diffusers-v3

like 6

Running

The name of Spaces is SDXL, a remnant from the old days, but it's not limited to SDXL.
However, free CPU space is not powerful enough to convert FLUX...

do you think I can run this locally? I have 64GB RAM and a 4090 GPU 24GB VRAM.

This GPU is more than three times more powerful than mine...
Even with that, FLUX won't work unless you quantize, but if you do quantize it will run smoothly.
An example of quantization is shown below.
Generally, fine-tuning requires performance that is several levels higher than inference, but it seems that there are now quite a few ways to tune it without using up too much GPU power.

You can convert to the Diffusers format by running the following script in Python. The basic idea is the same as the one used in Spaces.

for FLUX quantization

https://huggingface.co/docs/diffusers/main/en/quantization/bitsandbytes

for FLUX fine-tuning

https://fluxai.dev/blog/tutorial/2024-09/29-fine-tuning-flux-ai-with-10gb-vram
https://civitai.com/articles/7009/flux1-d-lora-on-a-12-gb-vram-train-settings-kohyass-web-ui
https://github.com/Nerogar/OneTrainer

Conversion script

from diffusers import FluxTransformer2DModel, FluxPipeline
from huggingface_hub import hf_hub_download
import torch
import gc
from pathlib import Path
import re
import urllib

base_model = "camenduru/FLUX.1-dev-diffusers"
model_url = "https://huggingface.co/datasets/John6666/flux1-backup-202502/blob/main/acornIsSpinningFLUX_aisFluxV15.safetensors"

def split_hf_url(url: str):
    try:
        s = list(re.findall(r'^(?:https?://huggingface.co/)(?:(datasets|spaces)/)?(.+?/.+?)/\w+?/.+?/(?:(.+)/)?(.+?.\w+)(?:\?download=true)?$', url)[0])
        if len(s) < 4: return "", "", "", ""
        repo_id = s[1]
        if s[0] == "datasets": repo_type = "dataset"
        elif s[0] == "spaces": repo_type = "space"
        else: repo_type = "model"
        subfolder = urllib.parse.unquote(s[2]) if s[2] else None
        filename = urllib.parse.unquote(s[3])
        return repo_id, filename, subfolder, repo_type
    except Exception as e:
        print(e)

model_url = model_url.replace("/resolve/main/", "/blob/main/").replace("?download=true", "")
repo_id, filename, subfolder, repo_type = split_hf_url(model_url)
model_file = hf_hub_download(repo_id=repo_id, filename=filename, repo_type=repo_type)

transformer = FluxTransformer2DModel.from_single_file(model_file, subfolder="transformer", torch_dtype=torch.bfloat16, config=base_model)
pipe = FluxPipeline.from_pretrained(base_model, transformer=transformer, torch_dtype=torch.bfloat16)
pipe.save_pretrained(Path(model_url).stem)

del pipe
torch.cuda.empty_cache()
gc.collect()

It worked like a charm, thank you!

I think I was unclear on my msg but I was wondering if I could run the diffusers conversion script locally, not planning to quantize or fine-tune but thanks for clarifying!

I'm using ai-toolkit trainer (which already quantizes) to create loras, so I'll use my local converted model as the checkpoint when training. I'll try that now.

Also, I could run the script locally (venv) and add these as requirements.txt (probably more than needed):

huggingface_hub
safetensors
transformers
accelerate
git+https://github.com/huggingface/diffusers
pytorch_lightning
peft
aria2
gdown
sentencepiece
torch==2.5.1
bitsandbytes
numpy<2
protobuf