Hugging Face's logo

taronaeo
/
Granite-3.0-1B-A400M-Instruct-BE-GGUF

Text Generation
GGUF
language
granite-3.0
mainframe
s390x
z15
z16
big-endian
Inference Endpoints
conversational
Model card Files Community
taronaeo commited on
Commit
85fa3df
·
verified ·
1 Parent(s): afdc598

refactor readme.md

Browse files

add gguf download links to provided files table

Files changed (1) hide show
  1. README.md +14 -14
README.md CHANGED
@@ -17,20 +17,20 @@ quantized_by: taronaeo
17
  This repository contains GGUF format model for [IBM Granite 3.0 1B Instruct](https://huggingface.co/ibm-granite/granite-3.0-1b-a400m-instruct). Every model has been verified to work on IBM z15 Mainframe.
18
 
19
  ### Provided Files
20
- | Name | Quant Method | Bits | Size | Use Case |
21
- |----------------------------------------------|--------------|------|------|------------------------------------------------------------------------|
22
- | granite-3.0-1b-a400m-instruct-be.Q2_K.gguf | Q2_K | 2 | 489M | smallest, significant quality loss - not recommended for most purposes |
23
- | granite-3.0-1b-a400m-instruct-be.Q3_K_S.gguf | Q3_K_S | 3 | 571M | very small, high quality loss |
24
- | granite-3.0-1b-a400m-instruct-be.Q3_K_M.gguf | Q3_K_M | 3 | 629M | very small, high quality loss |
25
- | granite-3.0-1b-a400m-instruct-be.Q3_K_L.gguf | Q3_K_L | 3 | 679M | small, substantial quality loss |
26
- | granite-3.0-1b-a400m-instruct-be.Q4_0.gguf | Q4_0 | 4 | 733M | legacy; small, very high quality loss - prefer using Q3_K_M |
27
- | granite-3.0-1b-a400m-instruct-be.Q4_K_S.gguf | Q4_K_S | 4 | 739M | small, greater quality loss |
28
- | granite-3.0-1b-a400m-instruct-be.Q4_K_M.gguf | Q4_K_M | 4 | 784M | medium, balanced quality - recommended |
29
- | granite-3.0-1b-a400m-instruct-be.Q5_0.gguf | Q5_0 | 5 | 886M | legacy; medium, balanced quality - prefer using Q4_K_M |
30
- | granite-3.0-1b-a400m-instruct-be.Q5_K_S.gguf | Q5_K_S | 5 | 886M | large, low quality loss - recommended |
31
- | granite-3.0-1b-a400m-instruct-be.Q5_K_M.gguf | Q5_K_M | 5 | 913M | large, very low quality loss - recommended |
32
- | granite-3.0-1b-a400m-instruct-be.Q6_K.gguf | Q6_K | 6 | 1.1G | very large, extremely low quality loss |
33
- | granite-3.0-1b-a400m-instruct-be.Q8_0.gguf | Q8_0 | 8 | 1.4G | very large, extremely low quality loss - not recommended |
34
 
35
  # Original model card: Granite-3.0-1B-A400M-Instruct
36
 
 
17
  This repository contains GGUF format model for [IBM Granite 3.0 1B Instruct](https://huggingface.co/ibm-granite/granite-3.0-1b-a400m-instruct). Every model has been verified to work on IBM z15 Mainframe.
18
 
19
  ### Provided Files
20
+ | Name | Quant Method | Bits | Size | Use Case |
21
+ |------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|------|------|------------------------------------------------------------------------|
22
+ | [granite-3.0-1b-a400m-instruct-be.Q2_K.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q2_K.gguf) | Q2_K | 2 | 489M | smallest, significant quality loss - not recommended for most purposes |
23
+ | [granite-3.0-1b-a400m-instruct-be.Q3_K_S.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q3_K_S.gguf) | Q3_K_S | 3 | 571M | very small, high quality loss |
24
+ | [granite-3.0-1b-a400m-instruct-be.Q3_K_M.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q3_K_M.gguf) | Q3_K_M | 3 | 629M | very small, high quality loss |
25
+ | [granite-3.0-1b-a400m-instruct-be.Q3_K_L.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q3_K_L.gguf) | Q3_K_L | 3 | 679M | small, substantial quality loss |
26
+ | [granite-3.0-1b-a400m-instruct-be.Q4_0.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q4_0.gguf) | Q4_0 | 4 | 733M | legacy; small, very high quality loss - prefer using Q3_K_M |
27
+ | [granite-3.0-1b-a400m-instruct-be.Q4_K_S.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q4_K_S.gguf) | Q4_K_S | 4 | 739M | small, greater quality loss |
28
+ | [granite-3.0-1b-a400m-instruct-be.Q4_K_M.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q4_K_M.gguf) | Q4_K_M | 4 | 784M | medium, balanced quality - recommended |
29
+ | [granite-3.0-1b-a400m-instruct-be.Q5_0.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q5_0.gguf) | Q5_0 | 5 | 886M | legacy; medium, balanced quality - prefer using Q4_K_M |
30
+ | [granite-3.0-1b-a400m-instruct-be.Q5_K_S.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q5_K_S.gguf) | Q5_K_S | 5 | 886M | large, low quality loss - recommended |
31
+ | [granite-3.0-1b-a400m-instruct-be.Q5_K_M.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q5_K_M.gguf) | Q5_K_M | 5 | 913M | large, very low quality loss - recommended |
32
+ | [granite-3.0-1b-a400m-instruct-be.Q6_K.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q6_K.gguf) | Q6_K | 6 | 1.1G | very large, extremely low quality loss |
33
+ | [granite-3.0-1b-a400m-instruct-be.Q8_0.gguf](https://huggingface.co/taronaeo/Granite-3.0-1B-A400M-Instruct-BE-GGUF/blob/main/granite-3.0-1b-a400m-instruct-be.Q8_0.gguf) | Q8_0 | 8 | 1.4G | very large, extremely low quality loss - not recommended |
34
 
35
  # Original model card: Granite-3.0-1B-A400M-Instruct
36