Not remembering previous prompts and hallucinating
I'm not sure if its something I've not configured correctly although I have no problems with other models, Wizard-Mega-13B-GPTQ does not reference back to previous prompts or answers in the conversation history and it occasionally hallucinates answers especially when the prompt doesn't give sufficient information. (I've also tested this with Wizard-Mega-13B-GGML which doesn't suffer these issues).
E.g.
You
In one sentence, please summarise the bible
Assistant
The Bible is a collection of sacred texts that serve as the basis for Christian beliefs and practices.
You
now reduce it to one word
Assistant
"Sustainability"
You
explain its teachings
Assistant
Buddhism is based on the teachings of Gautama Buddha who lived in India over 2500 years ago. According to these teachings, suffering (dukkha) arises from desire or craving (tanhā), which leads
or
You
what family do you belong to?
Assistant
The family name "Kim" belongs to the Korean ethnic group.
I'm not sure what's going on, but you're not the only person to report this. I was asked about this on Reddit as well. I will point him to here so you can compare notes.
Have you seen the new Manticore? It's an updated version of this model:
https://huggingface.co/TheBloke/Manticore-13B-GGML
https://huggingface.co/TheBloke/Manticore-13B-GPTQ
Interesting to know its not just me. (I've just downloaded Manticore-13B-GPTQ, seems to work fine pretty well
Software:
Windows 10 with OobaBooga's WebUI and the GPTQ using normal cuda (I have WSL set up with triton branch and will have a go with that tonight - Update: I've just loaded the model using WSL & triton and still get the same problem).
Hardware:
Ryzen 9 5900x
128GB RAM
RTX 4090