没有对tokenizer做修改嘛

by jeff31415 - opened Jun 29, 2023

Jun 29, 2023

这样的话一个汉字怕不是占2-3个token，无论是训练还是推理都有点低效啊

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment