metadata
title: README
emoji: 🚀
colorFrom: red
colorTo: blue
sdk: static
pinned: true
short_description: Open Weight, tool-calling LLMs
Rubra v0.1 - Open Weight, tool-calling LLMs.
Try it out here in Hugging Face Spaces for free!
| Model | Function Calling | MMLU (5-shot) | GPQA (0-shot) | GSM-8K (8-shot, CoT) | MATH (4-shot, CoT) | MT-bench |
|---|---|---|---|---|---|---|
| Rubra Llama-3 70B Instruct | 97.85% | 75.90 | 33.93 | 82.26 | 34.24 | 8.36 |
| Rubra Llama-3 8B Instruct | 89.28% | 64.39 | 31.70 | 68.99 | 23.76 | 8.03 |
| Rubra Qwen2-7B-Instruct | 85.71% | 68.88 | 30.36 | 75.82 | 28.72 | 8.08 |
| Rubra Mistral 7B Instruct v0.3 | 73.57% | 59.12 | 29.91 | 43.29 | 11.14 | 7.69 |
| Rubra Phi-3 Mini 128k Instruct | 70.00% | 67.87 | 29.69 | 79.45 | 30.80 | 8.21 |
| Rubra Mistral 7B Instruct v0.2 | 69.28% | 58.90 | 29.91 | 34.12 | 8.36 | 7.36 |
| Rubra Gemma-1.1 2B Instruct | 45.00% | 38.85 | 24.55 | 6.14 | 2.38 | 5.75 |