sentence-transformers

/

all-MiniLM-L6-v2

like 2.55k

Follow

Sentence Transformers 707

Done! See https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/tree/main/onnx
Also, this describes the usage: https://sbert.net/docs/sentence_transformer/usage/efficiency.html

@tomaarsen can you please provide a short quick way to create the openvino and quantized into an existing repo?

• given a model: https://huggingface.co/intfloat/multilingual-e5-small
• convert the model into openvino -> quantize
• push to a different hg repo

I'm running into issue.
This is how i do:

• download the model locally via

fast_model = SentenceTransformer('intfloat/multilingual-e5-small', backend="openvino")

• then it will not find the xml file, and will export the model to OpenVINO.
• then quantize and upload the model into my repo fails

export_static_quantized_openvino_model(
    fast_model,
    quantization_config=None,
    model_name_or_path="my-repo/multilingual-e5-small-openvino",
    push_to_hub=True,
    create_pr=True,
)

I have an Intel CPU with enough memory:
Issue:

[CPU] Add node with name '__module.embeddings/aten::add/Add' Exception from src\plugins\intel_cpu\src\shape_inference\custom\eltwise.cpp:45:
Eltwise shape infer input shapes dim index: 1 mismatch