Quantization made by Richard Erkhov.

pygmalion-1.3b - GGUF

Model creator: https://huggingface.co/PygmalionAI/
Original model: https://huggingface.co/PygmalionAI/pygmalion-1.3b/

Name	Quant method	Size
pygmalion-1.3b.Q2_K.gguf	Q2_K	0.53GB
pygmalion-1.3b.IQ3_XS.gguf	IQ3_XS	0.59GB
pygmalion-1.3b.IQ3_S.gguf	IQ3_S	0.61GB
pygmalion-1.3b.Q3_K_S.gguf	Q3_K_S	0.61GB
pygmalion-1.3b.IQ3_M.gguf	IQ3_M	0.66GB
pygmalion-1.3b.Q3_K.gguf	Q3_K	0.71GB
pygmalion-1.3b.Q3_K_M.gguf	Q3_K_M	0.71GB
pygmalion-1.3b.Q3_K_L.gguf	Q3_K_L	0.77GB
pygmalion-1.3b.IQ4_XS.gguf	IQ4_XS	0.74GB
pygmalion-1.3b.Q4_0.gguf	Q4_0	0.77GB
pygmalion-1.3b.IQ4_NL.gguf	IQ4_NL	0.78GB
pygmalion-1.3b.Q4_K_S.gguf	Q4_K_S	0.78GB
pygmalion-1.3b.Q4_K.gguf	Q4_K	0.85GB
pygmalion-1.3b.Q4_K_M.gguf	Q4_K_M	0.85GB
pygmalion-1.3b.Q4_1.gguf	Q4_1	0.85GB
pygmalion-1.3b.Q5_0.gguf	Q5_0	0.92GB
pygmalion-1.3b.Q5_K_S.gguf	Q5_K_S	0.92GB
pygmalion-1.3b.Q5_K.gguf	Q5_K	0.98GB
pygmalion-1.3b.Q5_K_M.gguf	Q5_K_M	0.98GB
pygmalion-1.3b.Q5_1.gguf	Q5_1	1.0GB
pygmalion-1.3b.Q6_K.gguf	Q6_K	1.08GB
pygmalion-1.3b.Q8_0.gguf	Q8_0	1.4GB

Original model description:

license: agpl-3.0 language:

en thumbnail: tags:
text generation
conversational inference: false

Pygmalion 1.3B

Model description

Pymalion 1.3B is a proof-of-concept dialogue model based on EleutherAI's pythia-1.3b-deduped.

Warning: This model is NOT suitable for use by minors. It will output X-rated content under certain circumstances.

Training data

The fine-tuning dataset consisted of 56MB of dialogue data gathered from multiple sources, which includes both real and partially machine-generated conversations.

Training procedure

Fine-tuning was done using ColossalAI (specifically, with a slightly modified version of their OPT fine-tune example) for around 11.4 million tokens over 5440 steps on a single 24GB GPU. The run took just under 21 hours.

Intended use

The easy way

We provide a notebook with a Gradio UI for playing around with the model without having to manually format inputs. This notebook can be found here.

The manual way

The model can be used as a regular text generation model, but it'll perform best if the input prompt adheres to the following format:

[CHARACTER]'s Persona: [A few sentences about the character you want the model to play]

[DIALOGUE HISTORY]
You: [Your input message here]
[CHARACTER]:

Where [CHARACTER] is, as you can probably guess, the name of the character you want the model to portray, and [DIALOGUE HISTORY] is chat history so the model can have some conversational context to draw from. Ideally it'll be pairs of messages like:

[CHARACTER]: [some dialogue here]
You: [your response to the dialogue above]

Apart from chat history, you can also just add example conversations in [DIALOGUE HISTORY] to show how the character should speak - ideally at the beginning, so it doesn't get confused as to what's conversation history vs. character definition.

Known issues

The model can get stuck repeating certain phrases, or sometimes even entire sentences.
- We believe this is due to that behavior being present in the training data itself, and plan to investigate and adjust accordingly for future versions.