metadata
license: apache-2.0
language:
- en
pipeline_tag: text-generation
Based on mistral 7b, Just for test. The Mistral-7B-v0.1 Large Language Model (LLM) is a pretrained generative text model with 7 billion parameters.
Model Architecture
Mistral-7B-v0.1 is a transformer model, with the following architecture choices:
Grouped-Query Attention
Sliding-Window Attention
Byte-fallback BPE tokenizer