balochiml
/

balochi-tokenizer

Model card Files Files and versions Community

balochi-tokenizer / src

Commit History

update trainer script

e8868b3

strickvl commited on Jun 3, 2023

update tokenizer script

57c829b

strickvl commited on Jun 3, 2023

add tokenization process

24469e0

strickvl commited on Jun 3, 2023

process and clean files

b3db2b8

strickvl commited on Jun 3, 2023

Get all text paths

470e696

strickvl commited on Jun 3, 2023

add data folder to gitignore

c9036ae

strickvl commited on Jun 3, 2023