Code is looking for 'modeling_flash_llama.py' on huggingface even though I have it in local folder

#2
by alexrider - opened

I have downloaded all the files, including 'modelling_flash_llama.py' and I am trying to load this model with autogptq.

model = AutoGPTQForCausalLM.from_quantized(subfolder_containing_model, ...)

However, the code is trying to find modelling_flash_llama.py from the togethercomputer repo even though I have the file in the local directory. Since I have no internet on the pc, I just get an error here "Could not locate the modelling_flash_llama.py inside togethercomputer//LLaMA-2-7B-32K". Is there any way to work around this?

Thanks for your help!

Sign up or log in to comment