Update README.md
Browse files
README.md
CHANGED
@@ -172,6 +172,8 @@ python -m executorch.examples.models.llama.export_llama \
|
|
172 |
-kv \
|
173 |
--use_sdpa_with_kv_cache \
|
174 |
-X \
|
|
|
|
|
175 |
--output_name="phi4-mini-8dq4w.pte"
|
176 |
```
|
177 |
|
@@ -200,7 +202,9 @@ Hello! I am Phi, an AI developed by Microsoft. I am not conscious in the way hum
|
|
200 |
|
201 |
Note: the runner does not currently recongize the stop token from Phi 4 Mini, so it generates text beyond when it should stop.
|
202 |
|
203 |
-
|
|
|
|
|
204 |
|
205 |
# Disclaimer
|
206 |
PyTorch has not performed safety evaluations or red teamed the quantized models. Performance characteristics, outputs, and behaviors may differ from the original models. Users are solely responsible for selecting appropriate use cases, evaluating and mitigating for accuracy, safety, and fairness, ensuring security, and complying with all applicable laws and regulations.
|
|
|
172 |
-kv \
|
173 |
--use_sdpa_with_kv_cache \
|
174 |
-X \
|
175 |
+
--xnnpack-extended-ops \
|
176 |
+
--metadata '{"get_bos_id":199999, "get_eos_ids":[200020,199999]}' \
|
177 |
--output_name="phi4-mini-8dq4w.pte"
|
178 |
```
|
179 |
|
|
|
202 |
|
203 |
Note: the runner does not currently recongize the stop token from Phi 4 Mini, so it generates text beyond when it should stop.
|
204 |
|
205 |
+
## Running in a mobile app
|
206 |
+
The model can be run in a mobile app. See [instructions](https://pytorch.org/executorch/main/llm/llama-demo-ios.html) for doing this on iOS.
|
207 |
+
TODO: add perf numbers, memory numbers, and screenshot from app
|
208 |
|
209 |
# Disclaimer
|
210 |
PyTorch has not performed safety evaluations or red teamed the quantized models. Performance characteristics, outputs, and behaviors may differ from the original models. Users are solely responsible for selecting appropriate use cases, evaluating and mitigating for accuracy, safety, and fairness, ensuring security, and complying with all applicable laws and regulations.
|