|
--- |
|
title: README |
|
emoji: 🚀 |
|
colorFrom: red |
|
colorTo: blue |
|
sdk: static |
|
pinned: true |
|
short_description: Open Weight, tool-calling LLMs |
|
--- |
|
# Rubra v0.1 - Open Weight, tool-calling LLMs. |
|
|
|
Try it out [here](https://huggingface.co/spaces/sanjay920/rubra-v0.1-function-calling) in Hugging Face Spaces for free! |
|
|
|
[https://docs.rubra.ai](https://docs.rubra.ai/) |
|
|
|
| Model | Function Calling | MMLU (5-shot) | GPQA (0-shot) | GSM-8K (8-shot, CoT) | MATH (4-shot, CoT) | MT-bench | |
|
|------------------------------------------|------------------|---------------|---------------|----------------------|--------------------|----------| |
|
| Rubra Llama-3 70B Instruct | 97.85% | 75.90 | 33.93 | 82.26 | 34.24 | 8.36 | |
|
| Rubra Llama-3 8B Instruct | 89.28% | 64.39 | 31.70 | 68.99 | 23.76 | 8.03 | |
|
| Rubra Qwen2-7B-Instruct | 85.71% | 68.88 | 30.36 | 75.82 | 28.72 | 8.08 | |
|
| Rubra Mistral 7B Instruct v0.3 | 73.57% | 59.12 | 29.91 | 43.29 | 11.14 | 7.69 | |
|
| Rubra Phi-3 Mini 128k Instruct | 70.00% | 67.87 | 29.69 | 79.45 | 30.80 | 8.21 | |
|
| Rubra Mistral 7B Instruct v0.2 | 69.28% | 58.90 | 29.91 | 34.12 | 8.36 | 7.36 | |
|
| Rubra Gemma-1.1 2B Instruct | 45.00% | 38.85 | 24.55 | 6.14 | 2.38 | 5.75 | |
