|
--- |
|
datasets: |
|
- PowerInfer/QWQ-LONGCOT-500K |
|
- PowerInfer/LONGCOT-Refine-500K |
|
base_model: |
|
- Qwen/Qwen2.5-3B-Instruct |
|
pipeline_tag: text-generation |
|
language: |
|
- en |
|
library_name: transformers |
|
tags: |
|
- llama-cpp |
|
- imatrix |
|
- gguf |
|
- IQ3_XXS |
|
- 3b |
|
- SmallThinker |
|
- qwen |
|
- llama-cpp |
|
- PowerInfer |
|
- code |
|
- math |
|
- chat |
|
- roleplay |
|
- text-generation |
|
- safetensors |
|
- nlp |
|
- code |
|
--- |
|
|
|
# roleplaiapp/SmallThinker-3B-Preview-IQ3_XXS-GGUF |
|
|
|
**Repo:** `roleplaiapp/SmallThinker-3B-Preview-IQ3_XXS-GGUF` |
|
**Original Model:** `SmallThinker-3B-Preview` |
|
**Organization:** `PowerInfer` |
|
**Quantized File:** `smallthinker-3b-preview-iq_xss-imat.gguf` |
|
**Quantization:** `GGUF` |
|
**Quantization Method:** `IQ3_XXS` |
|
**Use Imatrix:** `True` |
|
**Split Model:** `False` |
|
|
|
## Overview |
|
This is an imatrix GGUF IQ3_XXS quantized version of [SmallThinker-3B-Preview](https://huggingface.co/PowerInfer/SmallThinker-3B-Preview). |
|
|
|
## Quantization By |
|
I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models. |
|
I hope the community finds these quantizations useful. |
|
|
|
Andrew Webby @ [RolePlai](https://roleplai.app/) |
|
|