metadata

license: apache-2.0
pipeline_tag: text-generation
tags:
  - chat
  - mistral
  - roleplay
  - creative-writing
base_model:
  - nbeerbower/mistral-nemo-bophades-12B
  - anthracite-org/magnum-v2-12b
  - Sao10K/MN-12B-Lyra-v3
  - Gryphe/Pantheon-RP-1.6-12b-Nemo
language:
  - en

Quanting the Stardust V2 model. According to the model card this is a slightly different tune which you can see in the Usecase section from the original model card below. Highly recommend giving it a read-over and determining if you still want to try it before downloading. Either way, I intend to give it a shot.

This is the EXL2 8bpw version of this model. Find the original model here.
Find the 6bpw version here.
Find the 4bpw version here.

StarDust-12b-v2

Quants

GGUF: mradermacher/StarDust-12b-v2-GGUF
weighted/imatrix GGUF: mradermacher/StarDust-12b-v2-i1-GGUF
exl2: lucyknada/Luni_StarDust-12b-v2-exl2

Description | Usecase

The result of this merge is in my opinion a more vibrant and less generic sonnet inspired prose, it's able to be gentle and harsh where asked.
The v2 uses the non-kto magnum which tends to have less "claudeism" (making the story feel rather repetitive)
Note on Non-Kto: There is a very big gap between people preferring and disliking the KTO. To make things easier, you can still use Luni/StarDust-12b-v1 which has the KTO version.
In early testing users have reported a much better experience in longer roleplays and a abillity to add a creative touch to the stable experiencve.

Just like with v1:

This model is intended to be used as a Role-playing model.
Its direct conversational output is... I can't even say it's luck, it's just not made for it.
Extension to Conversational output: The Model is designed for roleplay, direct instructing or general purpose is NOT recommended.

Initial Feedback

Initial feedback has proven the model to be a solid "go-to" choice for creative storywriting
The prose has been certified as "amazing" with many making it their default model.

Prompting

ChatML has proven to be the BEST choice.

Both Mistral and ChatML should work though I had better results with ChatML: ChatML Example:

"""<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
"""

Merge Details

Merge Method

This model was merged using the DARE TIES merge method using Sao10K/MN-12B-Lyra-v3 as a base.

Models Merged

The following models were included in the merge:

Special Thanks

Special thanks to the SillyTilly and myself for helping me find the energy to finish this.