How does the MoE work?

#5
by PacmanIncarnate - opened

Can you explain how the gate picks between the two included models? Was there any additional merge system involved as well, or simple MoE?

Also, did you fine-tuned after the merge?

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment