Which hyperparameters to use for generation?

#2
by nv-bschifferer - opened

What system prompt, temperature, top_p, etc. parameters should we use for generating good result with NovaSky-AI/Sky-T1-32B-Flash ?

NovaSky org

Thanks for the question! We use the same system prompt as in the Preview model, which you can find here: https://github.com/NovaSky-AI/SkyThought/blob/f943eb96b84995fd60b27f5b0ad48b366b955764/skythought/skythought_evals/util/model_utils.py#L33

Unfortunately, we have not extensively explored settings for top_p and temperature to determine what is best, though simply setting temperature to 0.5 and top_p to 0.95 is a safe bet.

Sign up or log in to comment