Thanks for the ggml model - Docker integration
I've integrated it into a Docker build for "Open Llama-in-a-box" REST API server. Please keep the open-llama-3b-q5_1.bin file available for download. Full details here:
I will probably update the same repo when newer checkpoints are released, would that be alright?
The conversion itself is not anything complex. I will add the Makefile here someday.
I could maybe leave a branch for this specific one.
Sorry, didn't see your message.
@abetlen accepted my PR, so you can find the latest stable "open llama in-a-box" here:
https://github.com/abetlen/llama-cpp-python/tree/main/docker/open_llama
I just smoke tested your updated 3B repo and it looks to work without any changes with the above code. Once you have the 7B model converted, it a single line change in build.sh to d/l and package that, instead.