UnboundLocalError: cannot access local variable 'attn_weights' where it is not associated with a value

#17

by yyuxuan - opened Sep 17

Discussion

yyuxuan

Sep 17

When using model.generate(), met this error. Transformers == 4.40.0

yyuxuan

Sep 17

I believe this issue is related to flash_attention. In the Qwen2FlashAttention2 class (which inherits from Qwen2Attention) in modeling_qwen2.py, the attn_weights variable is not initialized beforehand.

yyuxuan changed discussion status to closed Sep 17

yyuxuan changed discussion status to open Sep 17

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment