TheBloke

/

Guanaco-13B-Uncensored-AWQ

like 3

I tried out the autoawq demo code as written in the description, and got the following error:

Traceback (most recent call last):
File "C:\Users\eric\Documents\Python\ml_test\awq_test.py", line 7, in
model = AutoAWQForCausalLM.from_quantized(model_name_or_path, fuse_layers=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\eric\miniconda3\envs\ml_test\Lib\site-packages\awq\models\auto.py", line 49, in from_quantized
return AWQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantized(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\eric\miniconda3\envs\ml_test\Lib\site-packages\awq\models\base.py", line 162, in from_quantized
load_checkpoint_in_model(
File "C:\Users\eric\miniconda3\envs\ml_test\Lib\site-packages\accelerate\utils\modeling.py", line 1063, in load_checkpoint_in_model
raise ValueError(f"{checkpoint} is not a folder containing a .index.json file.")
ValueError: C:\Users\eric.cache\huggingface\hub\models--TheBloke--Guanaco-13B-Uncensored-AWQ\snapshots\01a5af5abeb8cd68a1ce77a39aa5a7d28bedcd6a is not a folder containing a .index.json file.

Process finished with exit code 1

I checked and the model folder where it's looking has several json files in it but no .index.json file.