Sagicc/granite-8b-code-instruct-Q5_K_M-GGUF
This model was converted to GGUF format from ibm-granite/granite-8b-code-instruct
using llama.cpp after addded support for small Granite Code models in b3026 'llama.cpp release'.
Refer to the original model card for more details on the model.
For now only works with llama.cpp
Use with llama.cpp
Install llama.cpp through brew.
brew install ggerganov/ggerganov/llama.cpp
Invoke the llama.cpp server or the CLI.
CLI:
llama-cli --hf-repo Sagicc/granite-8b-code-instruct-Q5_K_M-GGUF --model granite-8b-code-instruct.Q5_K_M.gguf -p "You are an AI assistant"
Server:
llama-server --hf-repo Sagicc/granite-8b-code-instruct-Q5_K_M-GGUF --model granite-8b-code-instruct.Q5_K_M.gguf -c 2048
Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make && ./main -m granite-8b-code-instruct.Q5_K_M.gguf -n 128
- Downloads last month
- 2
Model tree for Sagicc/granite-8b-code-instruct-Q5_K_M-GGUF
Base model
ibm-granite/granite-8b-code-base-4kDatasets used to train Sagicc/granite-8b-code-instruct-Q5_K_M-GGUF
Evaluation results
- pass@1 on HumanEvalSynthesis(Python)self-reported57.900
- pass@1 on HumanEvalSynthesis(Python)self-reported52.400
- pass@1 on HumanEvalSynthesis(Python)self-reported58.500
- pass@1 on HumanEvalSynthesis(Python)self-reported43.300
- pass@1 on HumanEvalSynthesis(Python)self-reported48.200
- pass@1 on HumanEvalSynthesis(Python)self-reported37.200
- pass@1 on HumanEvalSynthesis(Python)self-reported53.000
- pass@1 on HumanEvalSynthesis(Python)self-reported42.700
- pass@1 on HumanEvalSynthesis(Python)self-reported52.400
- pass@1 on HumanEvalSynthesis(Python)self-reported36.600