speechless-starcoder2-7b
Code: https://github.com/uukuguy/speechless
Use the following dataset to fine-tune bigcode/starcoder2-7b in order to improve the model's reasoning and planning abilities.
Total 986k samples.
- teknium/OpenHermes-2.5
- TokenBender/python_eval_instruct_51k
- Spider
- codefuse-ai/Evol-instruction-66k
How to Prompt the Model
This model accepts the Alpaca instruction format.
For example:
You are an intelligent programming assistant.
### Instruction:
Implement a linked list in C++
### Response:
HumanEval
Metric | Value |
---|---|
humaneval-python |
lm-evaluation-harness
{'ARC (acc_norm)': ,
'HellaSwag (acc_norm)': ,
'MMLU (acc)': ,
'TruthfulQA (mc2)': ,
'Winoground (acc)': ,
'GSM8K (acc)': ,
'DROP (f1)': ,
'Open LLM Score': }
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | |
ARC (25-shot) | |
HellaSwag (10-shot) | |
MMLU (5-shot) | |
TruthfulQA (0-shot) | |
Winogrande (5-shot) | |
GSM8K (5-shot) | |
DROP (3-shot) |
- Downloads last month
- 806
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.