Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+gemma-ko-7b-instruct-v0.71-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,86 @@
+---
+language:
+- ko
+- en
+license: gemma
+library_name: transformers
+tags:
+- korean
+- gemma
+- pytorch
+- TensorBlock
+- GGUF
+base_model: lemon-mint/gemma-ko-7b-instruct-v0.71
+pipeline_tag: text-generation
+---
+
+<div style="width: auto; margin-left: auto; margin-right: auto">
+<img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;">
+</div>
+<div style="display: flex; justify-content: space-between; width: 100%;">
+    <div style="display: flex; flex-direction: column; align-items: flex-start;">
+        <p style="margin-top: 0.5em; margin-bottom: 0em;">
+            Feedback and support: TensorBlock's  <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a>
+        </p>
+    </div>
+</div>
+
+## lemon-mint/gemma-ko-7b-instruct-v0.71 - GGUF
+
+This repo contains GGUF format model files for [lemon-mint/gemma-ko-7b-instruct-v0.71](https://huggingface.co/lemon-mint/gemma-ko-7b-instruct-v0.71).
+
+The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
+
+<div style="text-align: left; margin: 20px 0;">
+    <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
+        Run them on the TensorBlock client using your local machine ↗
+    </a>
+</div>
+
+## Prompt template
+
+```
+<bos><start_of_turn>user
+{prompt}<end_of_turn>
+<start_of_turn>model
+```
+
+## Model file specification
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| [gemma-ko-7b-instruct-v0.71-Q2_K.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q2_K.gguf) | Q2_K | 3.242 GB | smallest, significant quality loss - not recommended for most purposes |
+| [gemma-ko-7b-instruct-v0.71-Q3_K_S.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q3_K_S.gguf) | Q3_K_S | 3.709 GB | very small, high quality loss |
+| [gemma-ko-7b-instruct-v0.71-Q3_K_M.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q3_K_M.gguf) | Q3_K_M | 4.069 GB | very small, high quality loss |
+| [gemma-ko-7b-instruct-v0.71-Q3_K_L.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q3_K_L.gguf) | Q3_K_L | 4.386 GB | small, substantial quality loss |
+| [gemma-ko-7b-instruct-v0.71-Q4_0.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q4_0.gguf) | Q4_0 | 4.668 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
+| [gemma-ko-7b-instruct-v0.71-Q4_K_S.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q4_K_S.gguf) | Q4_K_S | 4.700 GB | small, greater quality loss |
+| [gemma-ko-7b-instruct-v0.71-Q4_K_M.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q4_K_M.gguf) | Q4_K_M | 4.964 GB | medium, balanced quality - recommended |
+| [gemma-ko-7b-instruct-v0.71-Q5_0.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q5_0.gguf) | Q5_0 | 5.570 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
+| [gemma-ko-7b-instruct-v0.71-Q5_K_S.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q5_K_S.gguf) | Q5_K_S | 5.570 GB | large, low quality loss - recommended |
+| [gemma-ko-7b-instruct-v0.71-Q5_K_M.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q5_K_M.gguf) | Q5_K_M | 5.723 GB | large, very low quality loss - recommended |
+| [gemma-ko-7b-instruct-v0.71-Q6_K.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q6_K.gguf) | Q6_K | 6.529 GB | very large, extremely low quality loss |
+| [gemma-ko-7b-instruct-v0.71-Q8_0.gguf](https://huggingface.co/tensorblock/gemma-ko-7b-instruct-v0.71-GGUF/blob/main/gemma-ko-7b-instruct-v0.71-Q8_0.gguf) | Q8_0 | 8.454 GB | very large, extremely low quality loss - not recommended |
+
+
+## Downloading instruction
+
+### Command line
+
+Firstly, install Huggingface Client
+
+```shell
+pip install -U "huggingface_hub[cli]"
+```
+
+Then, downoad the individual model file the a local directory
+
+```shell
+huggingface-cli download tensorblock/gemma-ko-7b-instruct-v0.71-GGUF --include "gemma-ko-7b-instruct-v0.71-Q2_K.gguf" --local-dir MY_LOCAL_DIR
+```
+
+If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try:
+
+```shell
+huggingface-cli download tensorblock/gemma-ko-7b-instruct-v0.71-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'
+```

gemma-ko-7b-instruct-v0.71-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5e7327480ecb9271ec3f71cdd4f12c5587c19a768beeb528aa6660e56ceb7386
+size 3481447232

gemma-ko-7b-instruct-v0.71-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ded84a561147f1f079de4e02281c679b89e3004b3fd1fc75b5cc0d7b2759c8f1
+size 4709067584

gemma-ko-7b-instruct-v0.71-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac8aee4586fa3fabc0854d63357cf110e84f7327628ab231d191f22f65c2d0d3
+size 4369328960

gemma-ko-7b-instruct-v0.71-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:359d570c57a54e083e83731321d5c0b31554fdfc0067df0a645fca4fa4608325
+size 3982404416

gemma-ko-7b-instruct-v0.71-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:237c283731c9099577d16f2369bcadc17561ae643abda94dadd1e9b6551c6dbb
+size 5011843904

gemma-ko-7b-instruct-v0.71-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:042b61600fd05418129092bbdd86d46b45ff4f4e504d0a73b1119cb7d42c1cb7
+size 5329759040

gemma-ko-7b-instruct-v0.71-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ed09cbaa90aaa7253aca196fc2dc9316ba90a3e57527acb6a0a698cdd80cc19
+size 5046446912

gemma-ko-7b-instruct-v0.71-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5e6bafcf0612caad5c23daa1fd60b4d321439c9c119da273a17d56f116044b0
+size 5980728128

gemma-ko-7b-instruct-v0.71-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e12175bd08cb2daf2ad3d6ac0eaf59896b3947d070537d7eb70485f70727db4a
+size 6144502592

gemma-ko-7b-instruct-v0.71-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9882f309228c2d7ee6a12ca605d19a1446db703f495e66edcdc91b57ce90417d
+size 5980728128

gemma-ko-7b-instruct-v0.71-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c87dd9a3fd3b92e2c9ead03447f4cb7237f33a239e774d865753ff0fa6ee114
+size 7010167616

gemma-ko-7b-instruct-v0.71-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5cbfdb9962fcbb8f03ffdd7c9680b9148cf613b16f2eb6b2e3966151d74a1613
+size 9077844800