Example commmand to run:
python -m src.coreml_example --model-path-mf ./Llama-3.2-1B-4bits-MF.mlmodelc --model-path-head ./Llama-3.2-1B-HEAD-6Bits.mlmodelc/ --model-path-emb ./Llama-3.2-1B-EMB-16Bits.mlmodelc --prompt "Write a poem joke about Harry Potter" --max-tokens 400 --temp 1.2
You need to have set HF_TOKEN or provide on execution
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
HF Inference deployability: The model has no library tag.