d-matrix
commited on
Commit
•
3069bf4
1
Parent(s):
c250339
adding tokenizer
Browse files- tokenizer-2.json +0 -0
- tokenizer_config-2.json +1 -0
- vocab-2.json +0 -0
tokenizer-2.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
tokenizer_config-2.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"model_max_length": 1024}
|
vocab-2.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|