Post
2431
SwiftMistralCoreML
Hi Everyone,
I have created a Swift library to interact with Mistral 7B models in CoreML on macOS.
I hope you find it helpful.
https://github.com/cardona/SwiftMistralCoreML
An open-source Swift library that enables macOS and iOS projects to utilize the Mistral-Interact7B models (INT4 and upcoming FP16) in chat mode. This library includes a complete Swift implementation of the tokenizer and Byte Pair Encoding (BPE) encoder, providing an out-of-the-box solution for integrating advanced language models into your Swift applications.
Features
Full Swift Implementation: Includes tokenizer and BPE encoder written entirely in Swift.
CoreML Integration: Leverages Apple's CoreML framework to run Mistral-Interact7B models efficiently.
Multiple Decoding Strategies: Supports Greedy and Top-K sampling, with plans to add more strategies.
Chat Functionality: Designed to work in chat mode for interactive applications.
FP16 Support (Coming Soon): Future version will support FP16 models for improved performance.
Hi Everyone,
I have created a Swift library to interact with Mistral 7B models in CoreML on macOS.
I hope you find it helpful.
https://github.com/cardona/SwiftMistralCoreML
An open-source Swift library that enables macOS and iOS projects to utilize the Mistral-Interact7B models (INT4 and upcoming FP16) in chat mode. This library includes a complete Swift implementation of the tokenizer and Byte Pair Encoding (BPE) encoder, providing an out-of-the-box solution for integrating advanced language models into your Swift applications.
Features
Full Swift Implementation: Includes tokenizer and BPE encoder written entirely in Swift.
CoreML Integration: Leverages Apple's CoreML framework to run Mistral-Interact7B models efficiently.
Multiple Decoding Strategies: Supports Greedy and Top-K sampling, with plans to add more strategies.
Chat Functionality: Designed to work in chat mode for interactive applications.
FP16 Support (Coming Soon): Future version will support FP16 models for improved performance.
