Hi, Please someone tell me how exactly should I download the GGUF file from oobabooga txt gen ui.
I'm trying to use this ":" character and trying this:
TheBloke/Synthia-70B-v1.2b-GGUF
synthia-70b-v1.2b.Q4_K_M.gguf
Like, underneath.
However I keep getting an error.I have one more question. Namely, I have one 4090 and 64 gigs of RAM, will this model work at all for me? If so, what settings should I choose?
If you get an error, it would be helpful to post exactly what the error is.
You dont use the : character unless you are trying to specify a unique branch.
I can successfully download the model entering the values like this:
If you don't see the two fields, you may need to update your text-gen UI. It was added a month ago or so (I believe). I'm at commit: 40ffc3d687e119319bd50ad9b6c19ef098de7187.
You can also just download the model directly in your browser from huggingface and then copy it to your /models folder.
Oh! That make sense, I have only one field. Thanks!