AlessandroW commited on
Commit
2a1753a
·
verified ·
1 Parent(s): 1c9667c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -29,12 +29,15 @@ Resources and Technical Documentation:
29
  + [Phi-3 on Hugging Face](https://aka.ms/phi3-hf)
30
  + Phi-3 ONNX: [4K](https://aka.ms/phi3-mini-4k-instruct-onnx) and [128K](https://aka.ms/phi3-mini-128k-instruct-onnx)
31
 
32
- This repo provides GGUF files for the Phi-3 Mini-128K-Instruct model.
33
  | Name | Quant method | Bits | Size | Use case |
34
  | ---- | ---- | ---- | ---- | ----- |
35
  | [Phi-3-mini-128k-instruct-Q4_K_M.gguf](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 2.39 GB | medium, balanced quality - recommended |
36
- | [Phi-3-mini-128k-instruct-f16.gguf](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-f16.gguf) | None | 16 | 7.2 GB | minimal quality loss |
 
 
37
 
 
38
 
39
  ### License
40
 
 
29
  + [Phi-3 on Hugging Face](https://aka.ms/phi3-hf)
30
  + Phi-3 ONNX: [4K](https://aka.ms/phi3-mini-4k-instruct-onnx) and [128K](https://aka.ms/phi3-mini-128k-instruct-onnx)
31
 
32
+ This repo provides GGUF files and Llamafiles ([`d228e01d`](https://github.com/Mozilla-Ocho/llamafile/tree/d228e01d70a7b91bf04dbf63428646f3f173b888)) for the Phi-3 Mini-128K-Instruct model.
33
  | Name | Quant method | Bits | Size | Use case |
34
  | ---- | ---- | ---- | ---- | ----- |
35
  | [Phi-3-mini-128k-instruct-Q4_K_M.gguf](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 2.39 GB | medium, balanced quality - recommended |
36
+ | [Phi-3-mini-128k-instruct-Q4_K_M.llamafile](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.llamafile) | Q4_K_M | 4 | 2.4 GB | medium, balanced quality - recommended |
37
+ | [Phi-3-mini-128k-instruct-f16.gguf](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-f16.gguf) | None | 16 | 7.64 GB | minimal quality loss |
38
+ | [Phi-3-mini-128k-instruct-f16.llamafile](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-f16.llamafile) | None | 16 | 7.65 GB | minimal quality loss |
39
 
40
+ *Note:* When using the llamafile version make sure to specify the context size, e.g., `./Phi-3-mini-128k-instruct-Q4_K_M.llamafile -c 0 -p "your prompt"`.
41
 
42
  ### License
43