Public version of our 3b-model trained to perform 9 specific task in english and french with high accuracy. The benchmark bellow is done with our evaluation pipeline.
Model Fine-tune from Qwen/Qwen2.5-3B-Instruct with a specific LoRA adapter.
The prompt of our 9 specific task might follow in a next post. The goal of our team is to specialized small LLM on our clients specific needs while keeping the highest quality on general tasks.
Our 3b models compete with GPT-mini on those specific tasks.
+----------------------+---------+------------+------------+
| qwen3b | Overall | Team score | loads fail |
+----------------------+---------+------------+------------+
| answer_reformulation | 0.73 | 0.74 | 0.99 |
| query_reformulation | 0.83 | 0.99 | 0 |
| summarization | 0.92 | --- | 0 |
| keyword_extraction | 0.78 | --- | 0 |
| fill_in_generation | 0.86 | --- | 0 |
| keyword_update | 0.66 | 0.89 | 2 |
| gqa | 0.81 | 0.63 | 3 |
| true_false | 0.77 | --- | 0 |
| mcq | 0.86 | --- | 0 |
| Total | 0.8 | --- | 5.99 |
+----------------------+---------+------------+------------+
- Downloads last month
- 2,929
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.