TGI - response is an empty string
#6
by
p-christ
- opened
When i deploy this with TGI on Sagemaker it seems to work but all i ever get back is an empty string. Does anyone know why this might be?
Please use this model instead (TheBloke's is corrupted)
https://huggingface.co/casperhansen/mixtral-instruct-awq
@TheBloke
This model (when run on TGI) only generates <unk> (everything is okay with other AWQ-quantized (by you) mistral models. Can you fix it please please? 🙏🙏