Hugging Face's logo

stabilityai
/
stablelm-2-12b-chat-GGUF

GGUF
English
causal-lm
Inference Endpoints
conversational
Model card Files Community
1
jon-tow commited on
Commit
4101a30
•
1 Parent(s): 7c8c23e

Upload folder using huggingface_hub

Browse files
Files changed (9) hide show
  1. .gitattributes +1 -0
  2. README.md +7 -1
  3. stablelm-2-12b-chat-Q2_K.gguf +3 -0
  4. stablelm-2-12b-chat-Q3_K_M.gguf +3 -0
  5. stablelm-2-12b-chat-Q4_0.gguf +3 -0
  6. stablelm-2-12b-chat-Q4_K_M.gguf +3 -0
  7. stablelm-2-12b-chat-Q5_K_M.gguf +3 -0
  8. stablelm-2-12b-chat-Q6_K.gguf +3 -0
  9. stablelm-2-12b-chat-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -41,3 +41,4 @@ stablelm-2-12b-chat-q8_0.gguf filter=lfs diff=lfs merge=lfs -text
41
  stablelm-2-12b-chat-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
42
  stablelm-2-12b-chat-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
43
  stablelm-2-12b-chat-fp16.gguf filter=lfs diff=lfs merge=lfs -text
 
 
41
  stablelm-2-12b-chat-q6_k.gguf filter=lfs diff=lfs merge=lfs -text
42
  stablelm-2-12b-chat-q4_0.gguf filter=lfs diff=lfs merge=lfs -text
43
  stablelm-2-12b-chat-fp16.gguf filter=lfs diff=lfs merge=lfs -text
44
+ stablelm-2-12b-chat-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -35,7 +35,13 @@ extra_gated_fields:
35
  Make sure to install release [b2684](https://github.com/ggerganov/llama.cpp/releases/tag/b2684) or later.
36
 
37
  ```bash
38
- ./main -m stablelm-2-12b-chat-q4_k_m.gguf -c 4096 --temp 0.7 -p "Implement snake game using pygame"
 
 
 
 
 
 
39
  ```
40
 
41
  ## Model Details
 
35
  Make sure to install release [b2684](https://github.com/ggerganov/llama.cpp/releases/tag/b2684) or later.
36
 
37
  ```bash
38
+ ./main -m stablelm-2-12b-chat-q5_K_M.gguf -p "<|im_start|>user {PROMPT} <|im_end|><|im_start|>assistant"
39
+ ```
40
+
41
+ Use ChatML formatting for interactive chats:
42
+
43
+ ```bash
44
+ ./main -m stablelm-2-12b-chat-Q5_K_M.gguf -p {SYSTEM_PROMPT} -cml
45
  ```
46
 
47
  ## Model Details
stablelm-2-12b-chat-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d843f8def742683a0306b83199123520759778aadd41f81145e44a20718da950
3
+ size 4698894112
stablelm-2-12b-chat-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fc3dcec35e6236cba2966a5114077ca265182a6c5a264f6180e9b977a4a13a0
3
+ size 5993885472
stablelm-2-12b-chat-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d03a53ee6950fa378521cc811699813422b8486fa9aeeb6b0d52c5f81c3b721
3
+ size 6968897312
stablelm-2-12b-chat-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0da8ecc1024938c3215cf538d70b7e8a85ac29265da53ce30ec0e29656b154cd
3
+ size 7367642912
stablelm-2-12b-chat-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7997b28ab1cd46c2e4f0baa6a46a05f67631230042c0e7e5614d02cc05fb0d9e
3
+ size 8627900192
stablelm-2-12b-chat-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:76fa61a2d410c403e8c6d6e12dda56c50813518eecf978144fd052f4b6b1da32
3
+ size 9966923552
stablelm-2-12b-chat-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:68a057262d859c07bdff7cd02f94337657b4063806a8f626cd17fc11b4eba100
3
+ size 12907687712