Spaces:

yhavinga
/

dutch-tokenizer-arena

Running

dutch-tokenizer-arena / utils /compress_rate_util.py

add more tokenizer

c75633b 11 months ago

83 Bytes

	"""


	中文数据：clue superclue
	英文数据：glue cnn_dailymail gigaword

	"""