Problems with diffusers

#9
by Softology - opened

Using the diffusers code on the model card tab.
When the inference code runs I get the following errors. I could fix it by editing out these parameters, but it would be good to not to have to do that.

File "D:\Tests\AuraFlow\voc_auraflow\lib\site-packages\diffusers\models\normalization.py", line 87, in init
self.norm = FP32LayerNorm(embedding_dim, elementwise_affine=False, bias=False)
TypeError: LayerNorm.init() got an unexpected keyword argument 'bias'

File "D:\Tests\AuraFlow\voc_auraflow\lib\site-packages\diffusers\models\normalization.py", line 87, in init
self.norm = FP32LayerNorm(embedding_dim, elementwise_affine=False, bias=False)
TypeError: LayerNorm.init() got an unexpected keyword argument 'bias'

File "D:\Tests\AuraFlow\voc_auraflow\lib\site-packages\diffusers\models\attention_processor.py", line 179, in init
self.norm_q = FP32LayerNorm(dim_head, elementwise_affine=False, bias=False, eps=eps)
TypeError: LayerNorm.init() got an unexpected keyword argument 'bias'

File "D:\Tests\AuraFlow\voc_auraflow\lib\site-packages\diffusers\models\attention_processor.py", line 238, in init
self.norm_added_q = FP32LayerNorm(dim_head, elementwise_affine=False, bias=False, eps=eps)
TypeError: LayerNorm.init() got an unexpected keyword argument 'bias'

File "D:\Tests\AuraFlow\voc_auraflow\lib\site-packages\diffusers\models\transformers\auraflow_transformer_2d.py", line 198, in init
self.norm2 = FP32LayerNorm(dim, elementwise_affine=False, bias=False)
TypeError: LayerNorm.init() got an unexpected keyword argument 'bias'

File "D:\Tests\AuraFlow\voc_auraflow\lib\site-packages\diffusers\models\transformers\auraflow_transformer_2d.py", line 138, in init
self.norm2 = FP32LayerNorm(dim, elementwise_affine=False, bias=False)
TypeError: LayerNorm.init() got an unexpected keyword argument 'bias'

File "D:\Tests\AuraFlow\voc_auraflow\lib\site-packages\diffusers\models\attention_processor.py", line 1228, in call
hidden_states = F.scaled_dot_product_attention(
TypeError: scaled_dot_product_attention() got an unexpected keyword argument 'scale'

Once I removed the bias and scale parameters all is fine, but this will be an issue for anyone trying to run your example code.
This is the latest diffusers, ie
pip install git+https://github.com/huggingface/diffusers.git@bbd2f9d4e9ae70b04fedf65903fd1fb035437db4

@Softology Hi mate, could you please elaborate where you have removed those parameters as I am also trying to run it and facing issues with it. Thanks.

Filenames and line numbers shown in my post.
Just remove the ",bias=False" and scale parameters.

Got it, thanks.

burkaygur changed discussion status to closed

Please do not close this topic. It is not fixed yet.
My edits above are a work-around, not a fix. I am hoping the dev(s) can look at the actual problem and fix it.

fal org

Reopening and investigating.

burkaygur changed discussion status to open

If it helps, here are the pip installs I use within a new venv

python -m pip install --upgrade pip
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts wheel==0.43.0
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts git+https://github.com/huggingface/diffusers.git
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts transformers==4.42.4
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts accelerate==0.32.1
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts protobuf==5.27.2
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts sentencepiece==0.2.0
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts opencv_python==4.10.0.84
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts scipy==1.14.0
pip uninstall -y torch
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts torch==2.0.1+cu118 torchvision==0.15.2+cu118 --index-url https://download.pytorch.org/whl/cu118
pip uninstall -y charset-normalizer
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts charset-normalizer==3.3.0

and here is the inference.py script I test with

from diffusers import AuraFlowPipeline
import torch

pipeline = AuraFlowPipeline.from_pretrained(
"fal/AuraFlow",
torch_dtype=torch.float16
).to("cuda")

image = pipeline(
prompt="roses in the rain",
height=1024,
width=1024,
num_inference_steps=50,
generator=torch.Generator().manual_seed(2),
guidance_scale=3.5,
).images[0]

image.save("output.png")

fal org

Thanks for sharing. Really helpful!

fal org
edited Jul 19

Hi! Could you please upgrade the torch version you're using to at least 2.1? Looking at torch (2.0) docs, nn.LayerNorm does not expect bias as an argument, as does nn.functional's scaled_dot_product_attention. An upgrade to 2.1 or higher should fix this.
If I'm not wrong, your venv setup above suggests you're using torch==2.0.1.

Hi! Could you please upgrade the torch version you're using to at least 2.1? Looking at torch (2.0) docs, nn.LayerNorm does not expect bias as an argument, as does nn.functional's scaled_dot_product_attention. An upgrade to 2.1 or higher should fix this.
If I'm not wrong, your venv setup above suggests you're using torch==2.0.1.

Thank you! That was it. Updating torch to 2.3.0 fixed it.
For any other Windows users, here is the updated pip commands for setting up an environment for AuraFlow...

python -m pip install --upgrade pip
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts wheel==0.43.0
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts git+https://github.com/huggingface/diffusers.git@3f1411767bc0f1837adb6f289713807f18599db3
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts transformers==4.42.4
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts accelerate==0.32.1
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts protobuf==5.27.2
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts sentencepiece==0.2.0
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts opencv_python==4.10.0.84
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts scipy==1.14.0
pip uninstall -y torch
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts torch==2.3.0+cu118 torchvision --index-url https://download.pytorch.org/whl/cu118
pip uninstall -y charset-normalizer
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts charset-normalizer==3.3.0

Could you possibly look at my other issue with using a seed image?
https://huggingface.co/fal/AuraFlow/discussions/3
Any help there would be greatly appreciated.

Softology changed discussion status to closed

Sign up or log in to comment