prefill assistant responses
#16
by
alexsafayan
- opened
does this model support prefiling assistant responses?
e.g. like this:
chat = [
{
"role": "user", "content": [
{"type": "text", "content": "tell me the primary object in the image"},
{"type": "image", "image": image}
]
},
{
"role": "assistant", "content": [
{"type": "text", "content": "in one word, the primary object is: "}
]
}
]
I tried it and it appends a </s> to the end, causing a warning about padding. curious if anyone has tried this
You need to pass continue_final_message=True, more info in https://huggingface.co/docs/transformers/main/en/chat_templating#continuefinalmessage