---
license: apache-2.0
datasets:
- Skylion007/openwebtext
language:
- en
pipeline_tag: text-generation
---
A pretrained language model based on the Mistral 7B model, shrunk to about 248 million parameters. Minimal training was needed for this model, with only 250,000 examples over 125,000 steps required for convergence.