--- license: apache-2.0 language: - en - ar - zh - fr - ru - pt - es pipeline_tag: text-generation base_model: Qwen/Qwen2.5-14B tags: - chat library_name: transformers --- # Adept-14B-AWQ ## Introduction Adept-14B is a 14-billion-parameter generative AI model, leveraging Qwen2.5 14B and employing 4-bit AWQ quantization for efficiency. It is designed to provide organizations and developers with cutting-edge generative AI capabilities in a compact form, enabling high-quality instruction-following, complex reasoning, and tasks tailored for business applications. **From Qwen2.5:** - Significantly **more knowledge** and has greatly improved capabilities in **coding** and **mathematics**, thanks to our specialized expert models in these domains. - Significant improvements in **instruction following**, **generating long texts** (over 8K tokens), **understanding structured data** (e.g, tables), and **generating structured outputs** especially JSON. **More resilient to the diversity of system prompts**, enhancing role-play implementation and condition-setting for chatbots. - **Long-context Support** up to 128K tokens and can generate up to 8K tokens. - **Multilingual support** for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more. has the following features: - Type: Causal Language Models - Training Stage: Pretraining & Post-training - Architecture: transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias - Number of Parameters: 14.7B - Number of Paramaters (Non-Embedding): 13.1B - Number of Layers: 48 - Number of Attention Heads (GQA): 40 for Q and 8 for KV - Context Length: Full 131,072 tokens and generation 8192 tokens