Make 8 bit quantized model version

#14
by nonetrix - opened

It would be nice if there was a 8 bit gguff version as well for less compressed model

This comment has been hidden
This comment has been hidden

It would be nice if there was a 8 bit gguff version as well for less compressed model

where you able to run it with llama.cpp?
can you share some code example?

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment