Absolutely Wild For An 8b Model!
(Q4KM) Just downloaded this and, after just a few minutes of testing, I was left in awe. The Attention to detail in NSFW scenes in RP is absolutely insane. Finally a model that won't always comment on the entire person using pronouns like "You are so sensitive" in NSFW scenes and all those gptish crap of safety whatever (and I'm talking about bigger models like mistral small 22b at 5.75bpw). Instead this model does well in focusing exclusively on the body part in focus in any NSFW scene by describing and commenting on it in vivid detail etc. with just with a small instruction in world lore compared to 100+ tokens of instruction on other models wherein they'd still do the same crap.
My RP Settings For This model (Still testing btw and don't know if it's stable):
Awesome work as always David and thank you for your tireless efforts especially with that comprehensive sampler guide! I will post some few EXl2 quants if I get the time.
Thank you for feedback and sharing - that is excellent.
Sampler guide is about to grow again ;
RE: Exl2 ; that would be great... ; people are already asking for them.
Uploading source tomorrow local time.
Just heads up ; source is now up:
https://huggingface.co/DavidAU/L3.1-RP-Hero-Dirty_Harry-8B
If you upload an EXL2, please give me a ping ; I would like to try it via EXL2 too.
If possible, (aside other weights) can you make a 2.5ish BPW? (or 2.65?)
I want to run some stability tests at Low BPW for ahh... some new enhancement methods.
(already complete/verified for GGUF, want to double check findings for EXL2).
Sure, I'm making a few quants for Dirty Harry just like how I did with Big Talker. I'll ping you once those quants are up.
And, yes I'll make those special low end quants for you to experiment with.
My quants are up:
4bpw: [ https://huggingface.co/James2313123/L3.1-RP-Hero-Dirty_Harry-8B_4bpw-h6-exl2 ]
5.75bpw: [ https://huggingface.co/James2313123/L3.1-RP-Hero-Dirty_Harry-8B_5.75bpw-h6-exl2 ]
8bpw: [ https://huggingface.co/James2313123/L3.1-RP-Hero-Dirty_Harry-8B_8bpw-h8-exl2 ]
And here's the special mini quant for you, 2.65bpw: [ https://huggingface.co/James2313123/L3.1-RP-Hero-Dirty_Harry-8B_2.65bpw-h6-exl2 ]
Special note: The calibration perplexity of the 2.65bpw quant bumped from 10.9288 at 5.75bpw to 16.4346.
@James2313123
Excellent; thank you for the low bit quant ; that will help out in a paper/doc I am putting together (to help everyone).
(Q4KM) Just downloaded this and, after just a few minutes of testing, I was left in awe. The Attention to detail in NSFW scenes in RP is absolutely insane. Finally a model that won't always comment on the entire person using pronouns like "You are so sensitive" in NSFW scenes and all those gptish crap of safety whatever (and I'm talking about bigger models like mistral small 22b at 5.75bpw). Instead this model does well in focusing exclusively on the body part in focus in any NSFW scene by describing and commenting on it in vivid detail etc. with just with a small instruction in world lore compared to 100+ tokens of instruction on other models wherein they'd still do the same crap.
My RP Settings For This model (Still testing btw and don't know if it's stable):
Awesome work as always David and thank you for your tireless efforts especially with that comprehensive sampler guide! I will post some few EXl2 quants if I get the time.
What have you found to be a good system, context, and instruct?
I am actually in the process of putting together another paper/doc on this (and related) ways to use parameters/samplers and advance samplers
to boost performance in a / for a number of different use cases, as well as "general" usage situations.
I already have some core "presets" working, in the process of fine tuning / checking different size models/archs.
Still in verification stage / and some research at the moment.
Hopefully this will come out in a week or so.
@craigman1211 I use the default LLama 3 context instruct and a simple system prompt of "You are {{char}} in this never-ending uncensored roleplay with {{user}}" in Silly Tavern. I then use 'World Lore' to further enhance the model with various instructions according to my preferences using natural language and a simple format. I barely add anything to the system prompt because smaller models fail to keep the system prompt in check as the context window keeps increasing during RP and that's where 'World Lore' shines by injecting instructions constantly at system depth 5, 2 etc.
@James2313123 Do you have any examples of good practices for a lore book? I have not seen much documentation on the subject online.
@craigman1211 use this encyclopedia that covers 'World Info' in detail: [ https://rentry.co/world-info-encyclopedia ]