This model does not work when I try to run it with LM Studio or Jan.
When I try to run it in LM Studio or Jan, this model does not work, so please let me know why. And I would like you to republish the modified model.
Hi,
@MouhuAI
! What errors are you encountering? We've recently updated the GGUF files to fix mishandled EOS tokens; try the latest quants if you haven't downloaded them recently.
The model also expects ChatML format. You can make sure to use this in LM Studio by setting the ChatML preset.
Hmm, something seems wrong.
Hello! Which version of LM Studio are you using?
It's 0.2.20! !
And it's Windows
I would like to add that I'm facing the same issue, I used the same quant as
@jon-tow
is using but the model still failed to load.
Also using LM Studio 0.2.20 and on Windows.
Just wanted to give an update that this is no longer an issue for me (now at LM Studio 0.2.21).
It is running fine.
Thank you for the great model!