internlm2-chat-20b / tokenization_internlm2_fast.py

Commit History

fix no white space when using stream_chat with fast tokenizer
477d474

x54-729 commited on

update model weights
2481a80

ZwwWayne commited on

fix no white space when using stream_chat with fast tokenizer
19833d4

x54-729 commited on

fast tokenizer and stream_chat fix (#7)
d0438da
verified

RangiLyu x54-729 commited on