This model was converted to GGUF format from mlabonne/gemma-3-1b-it-abliterated using llama.cpp via the ggml.ai's all-gguf-same-where space.
Refer to the original model card for more details on the model.
Q4_K_M (Best balance of speed/quality)Q4_0 (Optimized for ARM CPUs)Q8_0 (Near-original quality)| π Download | π’ Type | π Notes |
|---|---|---|
| Download |  |
Basic quantization |
| Download |  |
Small size |
| Download |  |
Balanced quality |
| Download |  |
Better quality |
| Download |  |
Fast on ARM |
| Download |  |
Fast, recommended |
| Download |  β |
Best balance |
| Download |  |
Good quality |
| Download |  |
Balanced |
| Download |  |
High quality |
| Download |  π |
Very good quality |
| Download |  β‘ |
Fast, best quality |
| Download |  |
Maximum accuracy |
π‘ Tip: Use F16 for maximum precision when quality is critical
| Application | Description | Download Link |
|---|---|---|
| Llama.cpp | A fast and efficient inference engine for GGUF models. | GitHub Repository |
| Ollama | A streamlined solution for running LLMs locally. | Website |
| AnythingLLM | An AI-powered knowledge management tool. | GitHub Repository |
| Open WebUI | A user-friendly web interface for running local LLMs. | GitHub Repository |
| GPT4All | A user-friendly desktop application supporting various LLMs, compatible with GGUF models. | GitHub Repository |
| LM Studio | A desktop application designed to run and manage local LLMs, supporting GGUF format. | Website |
| GPT4All Chat | A chat application compatible with GGUF models for local, offline interactions. | GitHub Repository |
| Application | Description | Download Link |
|---|---|---|
| ChatterUI | A simple and lightweight LLM app for mobile devices. | GitHub Repository |
| Maid | Mobile Artificial Intelligence Distribution for running AI models on mobile devices. | GitHub Repository |
| PocketPal AI | A mobile AI assistant powered by local models. | GitHub Repository |
| Layla | A flexible platform for running various AI models on mobile devices. | Website |
| Application | Description | Download Link |
|---|---|---|
| Stable Diffusion | An open-source AI model for generating images from text. | GitHub Repository |
| Stable Diffusion WebUI | A web application providing access to Stable Diffusion models via a browser interface. | GitHub Repository |
| Local Dream | Android Stable Diffusion with Snapdragon NPU acceleration. Also supports CPU inference. | GitHub Repository |
| Stable-Diffusion-Android (SDAI) | An open-source AI art application for Android devices, enabling digital art creation. | GitHub Repository |
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Base model
google/gemma-3-12b-pt