Why using ununsual chat template?
#2
by
LoneRanger44
- opened
Hi, It seems that your models use unusual chat templates modified from ChatML. What is the point of this approach?
@LoneRanger44
We tried to be as close to ChatML as possible, but gemma's tokenizer doesn't have a <|im_end|> token, while adding any new token cause severe problem. So we use <eos> as stop signal instead.
Checkout SeaLLMs/SeaLLM-7B-v2.5-GGUF / seallm-7b-v2.5-chatml.Q4_K_M.gguf , where we change <eos> into <|im_end|> to allow ChatML to work. This will allow seamless integration with LM-studio.
Do unusual chat templates have any effect on finetune?
LoneRanger44
changed discussion status to
closed