AlessandroW
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -29,12 +29,15 @@ Resources and Technical Documentation:
|
|
29 |
+ [Phi-3 on Hugging Face](https://aka.ms/phi3-hf)
|
30 |
+ Phi-3 ONNX: [4K](https://aka.ms/phi3-mini-4k-instruct-onnx) and [128K](https://aka.ms/phi3-mini-128k-instruct-onnx)
|
31 |
|
32 |
-
This repo provides GGUF files for the Phi-3 Mini-128K-Instruct model.
|
33 |
| Name | Quant method | Bits | Size | Use case |
|
34 |
| ---- | ---- | ---- | ---- | ----- |
|
35 |
| [Phi-3-mini-128k-instruct-Q4_K_M.gguf](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 2.39 GB | medium, balanced quality - recommended |
|
36 |
-
| [Phi-3-mini-128k-instruct-
|
|
|
|
|
37 |
|
|
|
38 |
|
39 |
### License
|
40 |
|
|
|
29 |
+ [Phi-3 on Hugging Face](https://aka.ms/phi3-hf)
|
30 |
+ Phi-3 ONNX: [4K](https://aka.ms/phi3-mini-4k-instruct-onnx) and [128K](https://aka.ms/phi3-mini-128k-instruct-onnx)
|
31 |
|
32 |
+
This repo provides GGUF files and Llamafiles ([`d228e01d`](https://github.com/Mozilla-Ocho/llamafile/tree/d228e01d70a7b91bf04dbf63428646f3f173b888)) for the Phi-3 Mini-128K-Instruct model.
|
33 |
| Name | Quant method | Bits | Size | Use case |
|
34 |
| ---- | ---- | ---- | ---- | ----- |
|
35 |
| [Phi-3-mini-128k-instruct-Q4_K_M.gguf](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 2.39 GB | medium, balanced quality - recommended |
|
36 |
+
| [Phi-3-mini-128k-instruct-Q4_K_M.llamafile](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.llamafile) | Q4_K_M | 4 | 2.4 GB | medium, balanced quality - recommended |
|
37 |
+
| [Phi-3-mini-128k-instruct-f16.gguf](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-f16.gguf) | None | 16 | 7.64 GB | minimal quality loss |
|
38 |
+
| [Phi-3-mini-128k-instruct-f16.llamafile](https://huggingface.co/AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-f16.llamafile) | None | 16 | 7.65 GB | minimal quality loss |
|
39 |
|
40 |
+
*Note:* When using the llamafile version make sure to specify the context size, e.g., `./Phi-3-mini-128k-instruct-Q4_K_M.llamafile -c 0 -p "your prompt"`.
|
41 |
|
42 |
### License
|
43 |
|