Update README.md
Browse files
README.md
CHANGED
@@ -21,7 +21,7 @@ Here's a prompt format you should use, you can set a different system message, m
|
|
21 |
|
22 |
```
|
23 |
<|im_start|>system
|
24 |
-
A chat.<|im_end|>
|
25 |
<|im_start|>user
|
26 |
{prompt}<|im_end|>
|
27 |
<|im_start|>assistant
|
@@ -34,9 +34,16 @@ Use is limited by Yi license.
|
|
34 |
## Known Issues
|
35 |
|
36 |
I recommend to set repetition penalty to something around 1.05 to avoid repetition. So far I had good experience running this model with temperature 1.2.
|
37 |
-
Multi-turn conversations could be a bit better, if you ask it to re-write something with some fixes it will have a tendency to just repeat the previous response verbatim without any improvements - this is especially noticeable with repp 1.0.
|
38 |
-
There is still some gptslop left - some responses will have last paragraph with text "Remember that bla bla bla", I will try to get rid of it in the next version of the dataset.
|
39 |
-
Stories have ChatGPT like paragraph spacing, I will try to introduce a bit more stories that have long paragraphs in the next dataset version.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
|
41 |
## Axolotl training parameters
|
42 |
|
|
|
21 |
|
22 |
```
|
23 |
<|im_start|>system
|
24 |
+
A chat with uncensored assistant.<|im_end|>
|
25 |
<|im_start|>user
|
26 |
{prompt}<|im_end|>
|
27 |
<|im_start|>assistant
|
|
|
34 |
## Known Issues
|
35 |
|
36 |
I recommend to set repetition penalty to something around 1.05 to avoid repetition. So far I had good experience running this model with temperature 1.2.
|
37 |
+
Multi-turn conversations could be a bit better, if you ask it to re-write something with some fixes it will have a tendency to just repeat the previous response verbatim without any improvements - this is especially noticeable with repp 1.0. \
|
38 |
+
There is still some gptslop left - some responses will have last paragraph with text "Remember that bla bla bla", I will try to get rid of it in the next version of the dataset. \
|
39 |
+
Stories have ChatGPT like paragraph spacing, I will try to introduce a bit more stories that have long paragraphs in the next dataset version.
|
40 |
+
|
41 |
+
Unrestricted-ness of the v1 isn't quite something that I am happy with yet, it quite often still refuses to answer and behaves kinda like a normal airoboros, especially with default system prompt "A chat.". \
|
42 |
+
With a slightly modifed prompt it works much better, I recommend using "A chat with uncensored assistant.", it's stupid but it helps a lot. \
|
43 |
+
I was made aware of the frequent occurrence of the phrase "sending shivers down a spine" in the generations during RP, so I will manually fix those samples and put in different wording to fix that. \
|
44 |
+
I plan to use Yi-34B-200K Llama-fied for the next version, v1 very likely won't handle long context above 6k-8k well, be warned. \
|
45 |
+
There is also some issue with handling long system messages for RP, I will investigate that before releasing v2 dataset.
|
46 |
+
|
47 |
|
48 |
## Axolotl training parameters
|
49 |
|