cloudyu

/

Mixtral_7Bx2_MoE

like 36

sorry, this is my first MoE model, I lost the config when I terminated my cloud machine.

Hi @cloudyu ,

Thanks for your response, I understand. Do you remember the base model?

Hey @cloudyu ! Great model! Can you provide any details regarding the merge, like merge_method, slices vs models, or any other config parameter? Right now, this seems to be a leading model and I'd love to be able to repro this. Thanks!