|
--- |
|
license: apache-2.0 |
|
language: |
|
- en |
|
- ar |
|
- zh |
|
- fr |
|
- ru |
|
- pt |
|
- es |
|
pipeline_tag: text-generation |
|
base_model: Qwen/Qwen2.5-14B |
|
tags: |
|
- chat |
|
library_name: transformers |
|
--- |
|
|
|
# Adept-14B-AWQ |
|
|
|
## Introduction |
|
|
|
Adept-14B is a 14-billion-parameter generative AI model, leveraging Qwen2.5 14B and employing 4-bit AWQ quantization for efficiency. It is designed to provide organizations and developers with cutting-edge generative AI capabilities in a compact form, enabling high-quality instruction-following, complex reasoning, and tasks tailored for business applications. |
|
|
|
**From Qwen2.5:** |
|
|
|
- Significantly **more knowledge** and has greatly improved capabilities in **coding** and **mathematics**, thanks to our specialized expert models in these domains. |
|
- Significant improvements in **instruction following**, **generating long texts** (over 8K tokens), **understanding structured data** (e.g, tables), and **generating structured outputs** especially JSON. **More resilient to the diversity of system prompts**, enhancing role-play implementation and condition-setting for chatbots. |
|
- **Long-context Support** up to 128K tokens and can generate up to 8K tokens. |
|
- **Multilingual support** for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more. |
|
|
|
has the following features: |
|
- Type: Causal Language Models |
|
- Training Stage: Pretraining & Post-training |
|
- Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias |
|
- Number of Parameters: 14.7B |
|
- Number of Paramaters (Non-Embedding): 13.1B |
|
- Number of Layers: 48 |
|
- Number of Attention Heads (GQA): 40 for Q and 8 for KV |
|
- Context Length: Full 131,072 tokens and generation 8192 tokens |
