Emphasis DFSM - Nexesenex/kobold.cpp
@Nexesenex
- Heya, Nex! Regarding Emphasis DFSM, with the SillyTavern Grammar String set to emphasisfsm "-100"/emphasisfsm "-20", I see errors like these printed:
CtxLimit: 3080/10240, Process:0.01s (8.0ms/T = 125.00T/s), Generate:14.14s (47.0ms/T = 21.29T/s), Total:14.14s (21.28T/s)parse: error parsing grammar: expecting ::= at "-100"CtxLimit: 3062/10240, Process:8.65s (3.1ms/T = 321.35T/s), Generate:14.06s (49.7ms/T = 20.13T/s), Total:22.71s (12.46T/s)parse: error parsing grammar: expecting ::= at "-20"
Am I going about this incorrectly?
***
Welcome to KoboldCpp Frankenstein Fork1.70102
***
Based on LlamaCpp - Version b3398+4
***
Release date: 2024/07/15
***
Cuda mode compiled, if any: Cu12.2_Ar52617075_SMC2_DmmvX32Y1
***
GPU: NVIDIA GTX 1070Ti 8GB
With KoboldCpp Frankenstein Fork1.6840 - Based on LlamaCpp - Version b3235+59 - Release date: 2024/07/12 - Cuda mode compiled, if any: 12.2_F16int_SMC2_DmmvX64Y1 I still see the above errors parsing grammar, but afterwards I get:
csiz:0 par:8192 eilen:2779 np:0empcats_init: new grammar emphasisfsm "-100"
empcats_gen: ban bias: -100.000000
empcats_gen: emphasis indifferent tokens: 126802
empcats_gen: tokens for emphasis '"' '"': 1137
empcats_gen: tokens for emphasis '*' '*': 315
empcats_gen: always banned tokens: 2
empcats_gen: total tokens: 128256
What is the expected behavior/usage? Has anything changed?
Tagging some members that commented previously in relation to this topic, even if not exactly for this reason:
From: https://huggingface.co/LWDCLS/LLM-Discussions/discussions/12#668ce202a697b69d66345e3c
@Nexesenex Not sure if you are still able to provide any input on this? Should be able let me know.
Archiving for now.
Hey Lewdiculous.
Sorry, I skipped your message back in the days, due to inbox overflow. :X
For any question related to EmphasisFSM, it's better to ask on the github of the author : https://github.com/Yoshqu/koboldcpp-experimental.
Now that we have antislop in KCPP, I deactivated EmphasisFSM on my most recent branch.
His PR was helping the formatting of the chats using "" and ** to concatenate dialogues and actions.
Alrighty that makes sense, thanks Nex!