Saxo
/

Linkbricks-Horizon-AI-Korean-Superb-27B

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Saxo commited on Nov 16

Commit

4d38ecf

•

1 Parent(s): e808768

Update tokenizer_config.json

Files changed (1) hide show

tokenizer_config.json +1 -1

tokenizer_config.json CHANGED Viewed

@@ -2007,7 +2007,7 @@
   "pad_token": "<pad>",
   "sp_model_kwargs": {},
   "spaces_between_special_tokens": false,
-  "tokenizer_class": "PreTrainedTokenizerFast",
   "unk_token": "<unk>",
   "use_default_system_prompt": false
 }

   "pad_token": "<pad>",
   "sp_model_kwargs": {},
   "spaces_between_special_tokens": false,
+  "tokenizer_class": "GemmaTokenizer",
   "unk_token": "<unk>",
   "use_default_system_prompt": false
 }