metadata
license: mit
🔥 MoE-Mixtral-7B-8Expert
mixtral-8x7b is a Mixture-of-Expert (MoE) model. LLaMA2-Accessory has supported its inference and finetuning.
🚀 Features
With LLaMA2-Accessory, mixtral-8x7b enjoys the following features:
- Distributed MoE (namely instantiating experts on multiple processes/gpus)
- Load Balancing Loss
- Tensor Parallel and FSDP for efficiently training
- Distributed and/or quantized inference
🔥 Online Demo
We host a web demo at https://5e1109637f49baae47.gradio.live/, which shows a mixtral-8x7b model finetuned on evol-codealpaca-v1 and ultrachat_200k, with LoRA and Bias tuning. Please note that this is a temporary link, and we will update our official permanent link today.
💡 Tutorial
A detailed tutorial is available at https://llama2-accessory.readthedocs.io/en/latest/projects/mixtral-8x7b.html#