stduhpf/google-gemma-3-12b-it-qat-q4_0-gguf-small
Updated
โข
1.15k
โข
24
codito
codito
AI & ML interests
None yet
Recent Activity
liked
a model
3 days ago
stduhpf/google-gemma-3-12b-it-qat-q4_0-gguf-small
liked
a dataset
6 months ago
nvidia/HelpSteer2
Organizations
None yet
codito's activity
liked
a
model
3 days ago
liked
a
dataset
6 months ago
reacted to
tomaarsen's
post with ๐ฅ
6 months ago
Post
7127
๐ฃ Sentence Transformers v3.2.0 is out, marking the biggest release for inference in 2 years! 2 new backends for embedding models: ONNX (+ optimization & quantization) and OpenVINO, allowing for speedups up to 2x-3x AND Static Embeddings for 500x speedups at 10-20% accuracy cost.
1๏ธโฃ ONNX Backend: This backend uses the ONNX Runtime to accelerate model inference on both CPU and GPU, reaching up to 1.4x-3x speedup depending on the precision. We also introduce 2 helper methods for optimizing and quantizing models for (much) faster inference.
2๏ธโฃ OpenVINO Backend: This backend uses Intel their OpenVINO instead, outperforming ONNX in some situations on CPU.
Usage is as simple as
๐ Another major new feature is Static Embeddings: think word embeddings like GLoVe and word2vec, but modernized. Static Embeddings are bags of token embeddings that are summed together to create text embeddings, allowing for lightning-fast embeddings that don't require any neural networks. They're initialized in one of 2 ways:
1๏ธโฃ via Model2Vec, a new technique for distilling any Sentence Transformer models into static embeddings. Either via a pre-distilled model with
2๏ธโฃ Random initialization. This requires finetuning, but finetuning is extremely quick (e.g. I trained with 3 million pairs in 7 minutes). My final model was 6.6% worse than bge-base-en-v1.5, but 500x faster on CPU.
Full release notes: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.2.0
Documentation on Speeding up Inference: https://sbert.net/docs/sentence_transformer/usage/efficiency.html
1๏ธโฃ ONNX Backend: This backend uses the ONNX Runtime to accelerate model inference on both CPU and GPU, reaching up to 1.4x-3x speedup depending on the precision. We also introduce 2 helper methods for optimizing and quantizing models for (much) faster inference.
2๏ธโฃ OpenVINO Backend: This backend uses Intel their OpenVINO instead, outperforming ONNX in some situations on CPU.
Usage is as simple as
SentenceTransformer("all-MiniLM-L6-v2", backend="onnx"). Does your model not have an ONNX or OpenVINO file yet? No worries - it'll be autoexported for you. Thank me later ๐๐ Another major new feature is Static Embeddings: think word embeddings like GLoVe and word2vec, but modernized. Static Embeddings are bags of token embeddings that are summed together to create text embeddings, allowing for lightning-fast embeddings that don't require any neural networks. They're initialized in one of 2 ways:
1๏ธโฃ via Model2Vec, a new technique for distilling any Sentence Transformer models into static embeddings. Either via a pre-distilled model with
from_model2vec or with from_distillation where you do the distillation yourself. It'll only take 5 seconds on GPU & 2 minutes on CPU, no dataset needed.2๏ธโฃ Random initialization. This requires finetuning, but finetuning is extremely quick (e.g. I trained with 3 million pairs in 7 minutes). My final model was 6.6% worse than bge-base-en-v1.5, but 500x faster on CPU.
Full release notes: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.2.0
Documentation on Speeding up Inference: https://sbert.net/docs/sentence_transformer/usage/efficiency.html
- 1 reply
liked
a
Space
6 months ago
updated
3
models
7 months ago
upvoted
a
collection
7 months ago
reacted to
m-ric's
post with ๐ฅ
7 months ago
Post
3398
๐ฅ ๐๐ฐ๐๐ง ๐ซ๐๐ฅ๐๐๐ฌ๐๐ฌ ๐ญ๐ก๐๐ข๐ซ ๐.๐ ๐๐๐ฆ๐ข๐ฅ๐ฒ ๐จ๐ ๐ฆ๐จ๐๐๐ฅ๐ฌ: ๐๐๐ฐ ๐๐๐๐ ๐๐จ๐ซ ๐๐ฅ๐ฅ ๐ฌ๐ข๐ณ๐๐ฌ ๐ฎ๐ฉ ๐ญ๐จ ๐๐๐!
The Chinese LLM maker just dropped a flurry of different models, ensuring there will be a Qwen SOTA model for every application out there:
Qwen2.5: 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B
Qwen2.5-Coder: 1.5B, 7B, and 32B on the way
Qwen2.5-Math: 1.5B, 7B, and 72B.
And they didn't sleep: the performance is top of the game for each weight category!
๐๐๐ฒ ๐ข๐ง๐ฌ๐ข๐ ๐ก๐ญ๐ฌ:
๐ All models have ๐ญ๐ฎ๐ด๐ธ ๐๐ผ๐ธ๐ฒ๐ป ๐ฐ๐ผ๐ป๐๐ฒ๐ ๐ ๐น๐ฒ๐ป๐ด๐๐ต
๐ Models pre-trained on 18T tokens, even longer than the 15T of Llama-3
๐ช The flagship ๐ค๐๐ฒ๐ป๐ฎ.๐ฑ-๐ณ๐ฎ๐ ๐ถ๐ ~๐ฐ๐ผ๐บ๐ฝ๐ฒ๐๐ถ๐๐ถ๐๐ฒ ๐๐ถ๐๐ต ๐๐น๐ฎ๐บ๐ฎ-๐ฏ.๐ญ-๐ฐ๐ฌ๐ฑ๐, ๐ฎ๐ป๐ฑ ๐ต๐ฎ๐ ๐ฎ ๐ฏ-๐ฑ% ๐บ๐ฎ๐ฟ๐ด๐ถ๐ป ๐ผ๐ป ๐๐น๐ฎ๐บ๐ฎ-๐ฏ.๐ญ-๐ณ๐ฌ๐ ๐ผ๐ป ๐บ๐ผ๐๐ ๐ฏ๐ฒ๐ป๐ฐ๐ต๐บ๐ฎ๐ฟ๐ธ๐.
๐ซ๐ท On top of this, it ๐๐ฎ๐ธ๐ฒ๐ ๐๐ต๐ฒ #๐ญ ๐๐ฝ๐ผ๐ ๐ผ๐ป ๐บ๐๐น๐๐ถ๐น๐ถ๐ป๐ด๐๐ฎ๐น ๐๐ฎ๐๐ธ๐ so it might become my standard for French
๐ป Qwen2.5-Coder is only 7B but beats competing models up to 33B (DeeSeek-Coder 33B-Instruct). Let's wait for their 32B to come out!
๐งฎ Qwen2.5-Math sets a new high in the ratio of MATH benchmark score to # of parameters. They trained it by "aggregating more high-quality mathematical data, particularly in Chinese, from web sources, books, and codes across multiple recall cycles."
๐ Technical report to be released "very soon"
๐ All models have the most permissive license apache2.0, except the 72B models that have a custom license mentioning "you can use it for free EXCEPT if your product has over 100M users"
๐ค All models are available on the HF Hub! โก๏ธ Qwen/qwen25-66e81a666513e518adb90d9e
The Chinese LLM maker just dropped a flurry of different models, ensuring there will be a Qwen SOTA model for every application out there:
Qwen2.5: 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B
Qwen2.5-Coder: 1.5B, 7B, and 32B on the way
Qwen2.5-Math: 1.5B, 7B, and 72B.
And they didn't sleep: the performance is top of the game for each weight category!
๐๐๐ฒ ๐ข๐ง๐ฌ๐ข๐ ๐ก๐ญ๐ฌ:
๐ All models have ๐ญ๐ฎ๐ด๐ธ ๐๐ผ๐ธ๐ฒ๐ป ๐ฐ๐ผ๐ป๐๐ฒ๐ ๐ ๐น๐ฒ๐ป๐ด๐๐ต
๐ Models pre-trained on 18T tokens, even longer than the 15T of Llama-3
๐ช The flagship ๐ค๐๐ฒ๐ป๐ฎ.๐ฑ-๐ณ๐ฎ๐ ๐ถ๐ ~๐ฐ๐ผ๐บ๐ฝ๐ฒ๐๐ถ๐๐ถ๐๐ฒ ๐๐ถ๐๐ต ๐๐น๐ฎ๐บ๐ฎ-๐ฏ.๐ญ-๐ฐ๐ฌ๐ฑ๐, ๐ฎ๐ป๐ฑ ๐ต๐ฎ๐ ๐ฎ ๐ฏ-๐ฑ% ๐บ๐ฎ๐ฟ๐ด๐ถ๐ป ๐ผ๐ป ๐๐น๐ฎ๐บ๐ฎ-๐ฏ.๐ญ-๐ณ๐ฌ๐ ๐ผ๐ป ๐บ๐ผ๐๐ ๐ฏ๐ฒ๐ป๐ฐ๐ต๐บ๐ฎ๐ฟ๐ธ๐.
๐ซ๐ท On top of this, it ๐๐ฎ๐ธ๐ฒ๐ ๐๐ต๐ฒ #๐ญ ๐๐ฝ๐ผ๐ ๐ผ๐ป ๐บ๐๐น๐๐ถ๐น๐ถ๐ป๐ด๐๐ฎ๐น ๐๐ฎ๐๐ธ๐ so it might become my standard for French
๐ป Qwen2.5-Coder is only 7B but beats competing models up to 33B (DeeSeek-Coder 33B-Instruct). Let's wait for their 32B to come out!
๐งฎ Qwen2.5-Math sets a new high in the ratio of MATH benchmark score to # of parameters. They trained it by "aggregating more high-quality mathematical data, particularly in Chinese, from web sources, books, and codes across multiple recall cycles."
๐ Technical report to be released "very soon"
๐ All models have the most permissive license apache2.0, except the 72B models that have a custom license mentioning "you can use it for free EXCEPT if your product has over 100M users"
๐ค All models are available on the HF Hub! โก๏ธ Qwen/qwen25-66e81a666513e518adb90d9e
- 2 replies
liked
a
model
7 months ago
updated
a
model
7 months ago
liked
a
model
7 months ago
liked
a
model
8 months ago
updated
4
models
8 months ago
liked
a
Space
8 months ago
upvoted
a
paper
8 months ago
reacted to
gabrielmbmb's
post with ๐ฅ
8 months ago
Post
3590
Just dropped magpie-ultra-v0.1! The first open synthetic dataset generated with Llama 3.1 405B. Created with distilabel, it's our most advanced and compute-intensive pipeline to date. We made the GPUs of the cluster go brrrrr ๐
argilla/magpie-ultra-v0.1
Take it a look and tell us what you think! Probably, the models taking the most out of it are smol models ๐ค We will be improving the dataset in upcoming iterations!
argilla/magpie-ultra-v0.1
Take it a look and tell us what you think! Probably, the models taking the most out of it are smol models ๐ค We will be improving the dataset in upcoming iterations!