llmware
/

mistral-7b-instruct-v0.3-onnx

Model card Files Files and versions Community

mistral-7b-instruct-v0.3-onnx

mistral-7b-instruct-v0.3-onnx is an ONNX int4 quantized version of mistral-7b-instruct-v0.3, providing a fast inference implementation, optimized for AI PCs.

mistral-7b-instruct-v0.3 is the newest open source 7b instruct release from Mistral.

Model Description

Developed by: mistralai
Quantized by: llmware
Model type: mistral-7b-v0.3
Parameters: 7 billion
Model Parent: mistralai/Mistral-7B-Instruct-v0.3
Language(s) (NLP): English
License: Apache 2.0
Uses: General use
Quantization: int4

Model Card Contact

llmware on github

llmware website

Downloads last month: 5

Inference API

Inference API (serverless) has been turned off for this model.

Model tree for llmware/mistral-7b-instruct-v0.3-onnx

Base model

mistralai/Mistral-7B-v0.3

Finetuned

mistralai/Mistral-7B-Instruct-v0.3

Quantized

(118)

this model

Collection including llmware/mistral-7b-instruct-v0.3-onnx

Model Depot - ONNX

Leading Models packaged in ONNX format optimized for use with AI PCs • 20 items • Updated Oct 28