---
license: mit
license_link: >-
  https://huggingface.co/microsoft/Phi-3-mini-128k-instruct/resolve/main/LICENSE
language:
- en
pipeline_tag: text-generation
base_model: microsoft/Phi-3-mini-128k-instruct
tags:
- nlp
- code
---

## Model Summary

This repo provides the GGUF format for the Phi-3-Mini-128K-Instruct. 

*For more details check out the original model at [microsoft/Phi-3-mini-128k-instruct](https://huggingface.co/microsoft/Phi-3-mini-128k-instruct).*


The Phi-3-Mini-128K-Instruct is a 3.8 billion-parameter, lightweight, state-of-the-art open model trained using the Phi-3 datasets. This dataset includes both synthetic data and filtered publicly available website data, with an emphasis on high-quality and reasoning-dense properties. The model belongs to the Phi-3 family with the Mini version in two variants 4K and 128K which is the context length (in tokens) that it can support.

After initial training, the model underwent a post-training process that involved supervised fine-tuning and direct preference optimization to enhance its ability to follow instructions and adhere to safety measures. When evaluated against benchmarks that test common sense, language understanding, mathematics, coding, long-term context, and logical reasoning, the Phi-3 Mini-128K-Instruct demonstrated robust and state-of-the-art performance among models with fewer than 13 billion parameters. Resources and Technical Documentation:

Resources and Technical Documentation:

+ [Phi-3 Microsoft Blog](https://aka.ms/phi3blog-april)
+ [Phi-3 Technical Report](https://aka.ms/phi3-tech-report)
+ [Phi-3 on Azure AI Studio](https://aka.ms/phi3-azure-ai)
+ [Phi-3 on Hugging Face](https://aka.ms/phi3-hf)
+ Phi-3 ONNX: [4K](https://aka.ms/phi3-mini-4k-instruct-onnx) and [128K](https://aka.ms/phi3-mini-128k-instruct-onnx)

This repo provides GGUF files and Llamafiles ([`d228e01d`](https://github.com/Mozilla-Ocho/llamafile/tree/d228e01d70a7b91bf04dbf63428646f3f173b888)) for the Phi-3 Mini-128K-Instruct model. 
| Name | Quant method | Bits | Size | Use case |
| ---- | ---- | ---- | ---- | ----- |
| [Phi-3-mini-128k-instruct-Q4_K_M.gguf](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 2.39 GB | medium, balanced quality - recommended | 
| [Phi-3-mini-128k-instruct-Q4_K_M.llamafile](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.llamafile) | Q4_K_M | 4 | 2.4 GB | medium, balanced quality - recommended | 
| [Phi-3-mini-128k-instruct-f16.gguf](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-f16.gguf) | None | 16 | 7.64 GB | minimal quality loss |
| [Phi-3-mini-128k-instruct-f16.llamafile](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-f16.llamafile) | None | 16 | 7.65 GB | minimal quality loss |

*Note:* When using the llamafile version make sure to specify the context size, e.g., `./Phi-3-mini-128k-instruct-Q4_K_M.llamafile -c 0 -p "your prompt"`.

### License

The model is licensed under the [MIT license](https://huggingface.co/microsoft/phi-3-mini-128k/resolve/main/LICENSE).