InferenceIllusionist commited on
Commit
a04a545
1 Parent(s): 88d4c67

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +72 -1
README.md CHANGED
@@ -28,4 +28,75 @@ Please note importance matrix quantizations are a work in progress, IQ3 and abov
28
 
29
  <b>Tip:</b> Pick a size that can fit in your GPU while still allowing some room for context for best speed. You may need to pad this further depending on if you are running image gen or TTS as well.
30
 
31
- Original model card can be found [here](https://huggingface.co/rAIfle/0x01-8x7b-hf)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
  <b>Tip:</b> Pick a size that can fit in your GPU while still allowing some room for context for best speed. You may need to pad this further depending on if you are running image gen or TTS as well.
30
 
31
+ Original model card can be found [here](https://huggingface.co/rAIfle/0x01-8x7b-hf) and below. Check there for optimal settings.
32
+
33
+
34
+ # 0x01-8x7B-hf
35
+
36
+ ![grinning female android, cyberpunk, robotic, biomechanical, serial number "0x01"](https://files.catbox.moe/je2zar.png)
37
+
38
+ here we go again. multi-step merge, various models involved at various ratios with various methods.
39
+
40
+ this thing came to me in a fever dream when I was hung over, but after slightly tweaking the recipe it turned out surprisingly decent. using with the settings included.
41
+
42
+ ## Update:
43
+ The following settings have proved to work good too:
44
+ - Context: https://files.catbox.moe/q91rca.json
45
+ - Instruct: https://files.catbox.moe/2w8ja2.json
46
+ - Textgen: https://files.catbox.moe/s25rad.json
47
+
48
+
49
+ ## Constituent parts
50
+ ```yaml
51
+ # primordial_slop_a:
52
+ - model: mistralai/Mixtral-8x7B-v0.1+retrieval-bar/Mixtral-8x7B-v0.1_case-briefs
53
+ - model: mistralai/Mixtral-8x7B-v0.1+SeanWu25/Mixtral_8x7b_Medicine
54
+ - model: mistralai/Mixtral-8x7B-v0.1+SeanWu25/Mixtral_8x7b_WuKurtz
55
+ - model: mistralai/Mixtral-8x7B-v0.1+Epiculous/crunchy-onion-lora
56
+ - model: mistralai/Mixtral-8x7B-v0.1+maxkretchmer/gc-mixtral
57
+ # primordial_slop_b:
58
+ - model: Envoid/Mixtral-Instruct-ITR-8x7B
59
+ - model: crestf411/daybreak-mixtral-8x7b-v1.0-hf
60
+ - model: NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO
61
+ - model: orangetin/OpenHermes-Mixtral-8x7B
62
+ - model: mistralai/Mixtral-8x7B-Instruct-v0.1+idegroup/PhyAssistant
63
+ - model: ycros/crunchy-onion-nx
64
+ - model: jondurbin/bagel-dpo-8x7b-v0.2
65
+ - model: amoldwalunj/Mixtral-8x7B-Instruct-v0.1-legal_finetune_mixtral_32k
66
+ # primordial_slop_c: a+b
67
+ # primordial_slop_d:
68
+ - model: Sao10K/Sensualize-Mixtral-bf16
69
+ - model: Envoid/Mixtral-Instruct-ITR-DADA-8x7B
70
+ ```
71
+
72
+ # mergekit
73
+
74
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
75
+
76
+ ## Merge Details
77
+ ### Merge Method
78
+
79
+ This model was merged using the SLERP merge method.
80
+
81
+ ### Models Merged
82
+
83
+ The following models were included in the merge:
84
+ * ./primordial_slop_d
85
+ * ./primordial_slop_c
86
+
87
+ ### Configuration
88
+
89
+ The following YAML configuration was used to produce this model:
90
+
91
+ ```yaml
92
+ models:
93
+ - model: ./primordial_slop_c
94
+ - model: ./primordial_slop_d
95
+ merge_method: slerp
96
+ base_model: ./primordial_slop_c
97
+ parameters:
98
+ t:
99
+ - value: 0.33
100
+ dtype: float16
101
+
102
+ ```