gguf fail to be loaded on ollama and LM Studio

#21

by JackeyBee - opened Jul 29

Jul 29

When I try to load Phi-3-mini-4k-instruct-q4.gguf, it says "llama.cpp error: 'error loading model hyperparameters: key not found in model: phi3.attention.sliding_window'". I have spotted the issue created on ollama GH repository too: link

riedgar-ms

Jul 29

•

edited Jul 29

I'm seeing something similar with loading via llama-cpp-python. With another "'Llama' object has no attribute '_lora_adapter'" message. This has happened only with the latest llama-cpp-python release.

kaetemi

Jul 29

See https://github.com/ggerganov/llama.cpp/pull/8627

Updated gguf are required since this recent change.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment