|
--- |
|
language: |
|
- en |
|
- ru |
|
tags: |
|
- not-for-all-audiences |
|
- nsfw |
|
base_model: |
|
- LakoMoor/Silicon-Alice-7B |
|
license: cc-by-nc-4.0 |
|
inference: false |
|
library_name: transformers |
|
model_creator: LakoMoor |
|
model_name: Silicon-Alice-7B |
|
model_type: mistral |
|
--- |
|
|
|
# Silicon-Alice-7B-GGUF |
|
 |
|
|
|
## What's that? |
|
|
|
Silicon-Alice-7B-GGUF is a quantized model based on [Silicon-Masha-7B](https://huggingface.co/LakoMoor/Silicon-Alice-7B) aiming to be both strong in RP, be smart **and** understand Russian, that can follow character maps very well. This model understands Russian better than the previous one. It is suitable for RP/ERP and general use. It can be run on weak samovar using [llama.cpp](https://github.com/ggerganov/llama.cpp) or [koboldcpp](https://github.com/LostRuins/koboldcpp). |
|
|
|
## Prompt Template (Alpaca) |
|
|
|
I found the best SillyTavern results from using the Noromaid template but please try other templates! Let me know if you find anything good. |
|
|
|
SillyTavern config files: [Context](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/resolve/main/assets/context.json), [Instruct](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/resolve/main/assets/instruct.json). |
|
|
|
Additionally, here is my highly recommended [Text Completion preset](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/resolve/main/assets/MinP.json). You can tweak this by adjusting temperature up or dropping min p to boost creativity or raise min p to increase stability. You shouldn't need to touch anything else! |
|
|
|
``` |
|
Below is an instruction that describes a task. Write a response that appropriately completes the request. |
|
### Instruction: |
|
{prompt} |
|
### Response: |
|
``` |
|
|
|
## Provided files |
|
|
|
| Name | Quant method | Bits | Use case | |
|
| ---- | ---- | ---- | ----- | |
|
| [silicon-alice-7b.Q2_K.gguf](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/blob/main/silicon-alice-7b.Q2_K.gguf) | Q2_K | 2 | smallest, significant quality loss - not recommended for most purposes | |
|
| [silicon-alice-7b.Q3_K_M.gguf](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/blob/main/silicon-alice-7b.Q3_K_M.gguf) | Q3_K_M | 3 | very small, high quality loss | |
|
| [silicon-alice-7b.Q4_0.gguf](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/blob/main/silicon-alice-7b.Q4_0.gguf) | Q4_0 | 4 | legacy; small, very high quality loss - prefer using Q3_K_M | |
|
| [silicon-alice-7b.Q4_K_M.gguf](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/blob/main/silicon-alice-7b.Q4_K_M.gguf) | Q4_K_M | 4 | medium, balanced quality - recommended | |
|
| [silicon-alice-7b.Q5_0.gguf](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/blob/main/silicon-alice-7b.Q5_0.gguf) | Q5_0 | 5 | legacy; medium, balanced quality - prefer using Q4_K_M | |
|
| [silicon-alice-7b.Q5_K_M.gguf](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/blob/main/silicon-alice-7b.Q5_K_M.gguf) | Q5_K_M | 5 | large, very low quality loss - recommended | |
|
| [silicon-alice-7b.Q6_K.gguf](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/blob/main/silicon-alice-7b.Q6_K.gguf) | Q6_K | 6 | very large, extremely low quality loss | |
|
| [silicon-alice-7b.Q8_0.gguf](https://huggingface.co/LakoMoor/Silicon-Alice-7B-GGUF/blob/main/silicon-alice-7b.Q8_0.gguf) | Q8_0 | 8 | very large, extremely low quality loss - not recommended | |
|
|
|
## How run it ? |
|
[llama.cpp](https://github.com/ggerganov/llama.cpp) |
|
```shell |
|
./main -ngl 35 -m silicon-alice-7b.Q4_K_M.gguf --color -c 32768 --temp 0.4 --repeat_penalty 1.1 -n -1 -p "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n{system_message}\n### Instruction:{prompt}\n### Response:\n" |
|
``` |