Dampish commited on
Commit
6687143
·
1 Parent(s): baafcc3

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +48 -0
README.md ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-sa-4.0
3
+ ---
4
+
5
+ Model trained to be as helpful of an assistant as possible.
6
+
7
+ ### Data split percentage
8
+ 60% coding
9
+ 10% conversations
10
+ 20% instructions
11
+ 10% Roleplay
12
+
13
+ Now obviously the data contains other elements but this is the biggest part
14
+
15
+ ### Examples
16
+ ```
17
+ The prompt structure can be almost anything, this model has been trained on 0.6 million instruction, which is less than dante.
18
+ This model has however been trained on a much cleaner and organized dataset and has been retrained multiple times to reach as low
19
+ of a training loss as possible.
20
+
21
+ ## Examples.
22
+
23
+ "You are an AI assistant respond to human in a helpful manner.
24
+ HM: What were the causes for world war 2?
25
+ "
26
+
27
+ "Act like a detective from the 1900s, respond to mike in a helpful manner.
28
+ HM: What were the causes for world war 2?
29
+ "
30
+
31
+ The prompt also works with alpaca structure. I have purposefully trained it so that it should work like this.
32
+ EOS token is <|end|>.
33
+
34
+ Remember to tell it how it should act for best effect.
35
+ ```
36
+
37
+ ### More information
38
+ The base model is GPT-NeoX taken pretrained by redpajama.
39
+
40
+ We managed to reach 0.45 Validation loss with a 0.3 learning loss.
41
+
42
+ You are not allowed to use this for commerical purposes unless reaching an agreement with the creator @Dampish, @Dampish#3607 on discord.
43
+
44
+
45
+
46
+
47
+
48
+ The model can easily be further fine tuned to most languages.