metadata

language:
  - en
library_name: transformers
license: other
license_name: microsoft-terms-of-use
license_link: https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/resolve/main/LICENSE
tags:
  - text-generation-inference
  - phi3
  - awq
  - microsoft
extra_gated_heading: Access Microsoft on Hugging Face
extra_gated_prompt: >-
  To access Phi-3 on Hugging Face, you’re required to review and agree to
  Microsoft usage license. To do this, please ensure you’re logged-in to Hugging
  Face and click below. Requests are processed immediately.
extra_gated_button_content: Acknowledge license

phi-3-mini-4k-instruct-awq-4bit

phi-3-mini-4k-instruct-awq-4bit is a version of the Microsoft Phi 3 mini 4k Instruct model that was quantized using the AWQ method developed by Lin et al. (2023).

Please refer to the Original Phi 3 mini model card for details about the model preparation and training processes.

Dependencies

autoawq==0.2.5 – AutoAWQ was used to quantize the phi-3 model.
vllm==0.4.2 – vLLM was used to host models for benchmarking.