Which hyperparameters to use for generation?
#2
by
nv-bschifferer
- opened
What system prompt, temperature, top_p, etc. parameters should we use for generating good result with NovaSky-AI/Sky-T1-32B-Flash ?
Thanks for the question! We use the same system prompt as in the Preview model, which you can find here: https://github.com/NovaSky-AI/SkyThought/blob/f943eb96b84995fd60b27f5b0ad48b366b955764/skythought/skythought_evals/util/model_utils.py#L33
Unfortunately, we have not extensively explored settings for top_p and temperature to determine what is best, though simply setting temperature to 0.5 and top_p to 0.95 is a safe bet.