16K or 32K context length?

#4
by ddh0 - opened

Hello, does this model only support 16K, or can it be extended to 32K? Since the base model is 32K context length, I am confused why this one is 16K.

Thank you!

01-ai org

Hi ddh0,

Yes, you can extend the 16k chat model to 32k simply by changing max position embeddings in both config and during inference. But expect the response quality to degrade.

thanks,
Nuo

Cool, thank you for the clarification!

ddh0 changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment