brittlewis12/Memphis-CoT-3B-GGUF

Memphis-CoT-3B GGUF

Original model: Memphis-CoT-3B Model creator: euclaise

This repo contains GGUF format model files for euclaise’s Memphis-CoT-3B, updated for the latest training run as of 2/2/24.

Memphis-CoT is a finetune of StableLM 3b 4e1t on TinyCoT, along with reddit-instruct (subset to 5000 examples, excluding posts with brackets in the title) and a curated subset of oasst2.

What is GGUF?

GGUF is a file format for representing AI models. It is the third version of the format, introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. Converted using llama.cpp b2022 (8f8ddfc)

Prompt template:

{{system_message}}
### User:
{{prompt}}
### Assistant:

or Tiny CoT:

### User:
{{prompt}}
### Rationale:
[...]
### Answer:

---

Download & run with cnvrs on iPhone, iPad, and Mac!

cnvrs is the best app for private, local AI on your device:

• create & save Characters with custom system prompts & temperature settings
• download and experiment with any GGUF model you can find on HuggingFace!
• make it your own with custom Theme colors
• powered by Metal ⚡️ & Llama.cpp, with haptics during response streaming!
• try it out yourself today, on Testflight!
• follow cnvrs on twitter to stay up to date

---

Original Model Evaluations:

Model	Size	Data	Method	GSM8K (5-shot)	AGIEval (English/Nous subset, acc_norm)	BIG Bench Hard (CoT, few-shot*)
StableLM 3B Base	3B	Base	Base	2.05%	25.14%	36.75%
StableHermes 3B	3B	GPT	SFT	3.64%	24.31%	37.28%
MPT 7B Instruct	7B	Human+Anthropic	SFT	2.05%	24.12%	11.01%
OpenLLaMA 7B v2 open-instruct	7B	Human (nearly: ecqa is an exception)	SFT	8.64%	23.21%	29.84%
StableLM Zephyr 3B	3B	GPT	DPO	possibly contaminated (45.72%)	33.31%	0.91%
Memphis-CoT 3B	3B	Human	Self-teaching	13.8%	26.24%	38.24%

*5-shot, as performed automatically by LM Evaluation Harness bbh_cot_fewshot even with num_fewshot=0

Memphis outperforms other primarily-human-data models that are over twice its size, along with SFT models of its size, and trades with the Zephyr DPO model. That said, Zephyr uses synthetic data, and much more of it.