File size: 7,324 Bytes
7886d18
552a125
 
 
 
 
7886d18
 
419ba3c
 
 
 
679182a
7886d18
 
552a125
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7886d18
3a89443
c00b53e
 
c2aaa2d
2470754
735c8ad
 
 
 
 
 
aa84a25
05b1268
 
 
c00b53e
3a89443
7886d18
 
aad8879
7886d18
 
 
 
aad8879
7886d18
 
 
 
419ba3c
 
679182a
419ba3c
7886d18
16d91e6
7886d18
 
7c2c628
7886d18
 
 
 
419ba3c
b723c6f
1c29bfa
 
 
 
 
 
 
 
 
 
 
 
b723c6f
 
 
5db79d8
1c29bfa
 
 
 
 
 
 
 
 
 
 
 
5db79d8
b723c6f
 
1c29bfa
 
 
 
 
 
 
 
 
 
 
 
419ba3c
b723c6f
 
 
7886d18
 
 
 
2a9d0da
7886d18
 
 
 
 
2ad5639
 
735c8ad
 
aad8879
 
552a125
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
---
library_name: transformers
tags:
- mergekit
- merge
- llama
base_model:
- v000000/L3.1-8B-RP-Test-003-Task_Arithmetic
- v000000/L3.1-Niitorm-8B-t0.0001
- Sao10K/L3.1-8B-Niitama-v1.1
- arcee-ai/Llama-3.1-SuperNova-Lite
- akjindal53244/Llama-3.1-Storm-8B
- arcee-ai/Llama-Spark
- v000000/L3.1-8B-RP-Test-002-Task_Arithmetic
- grimjim/Llama-3-Instruct-abliteration-LoRA-8B
model-index:
- name: L3.1-Storniitova-8B
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: HuggingFaceH4/ifeval
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 78.17
      name: strict accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: BBH
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 30.81
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: hendrycks/competition_math
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 13.29
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 5.26
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 9.96
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 30.84
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=v000000/L3.1-Storniitova-8B
      name: Open LLM Leaderboard
---

# Llama-3.1-Storniitova-8B

Storniitova-8B is a RP/Instruct model built on the foundation of Llama-3.1-SuperNova-Lite, which is distilled from the 405B parameter variant of Llama-3.1

By only changing the vector tasks, I attempt to retain the full 405B distillation while learning roleplaying capabilties.

# (GGUF) mradermacher quants:

* [GGUFs](https://huggingface.co/mradermacher/L3.1-Storniitova-8B-GGUF)
* [GGUFs imatrix](https://huggingface.co/mradermacher/L3.1-Storniitova-8B-i1-GGUF)

# (GGUF) QuantFactory quants:
* [GGUFs](https://huggingface.co/QuantFactory/L3.1-Storniitova-8B-GGUF)

-----------------------------------------------------------------------------------------------------------

# merge

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit) and other proprietary tools.

## Merge Details
### Merge Method

This model was merged using the <b>SLERP, Task_Arithmetic and NEARSWAP</b> merge method.

### Models Merged

The following models were included in the merge:
* [v000000/L3.1-Niitorm-8B-t0.0001](https://huggingface.co/v000000/L3.1-Niitorm-8B-t0.0001)
* [akjindal53244/Llama-3.1-Storm-8B](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B)
* [arcee-ai/Llama-Spark](https://huggingface.co/arcee-ai/Llama-Spark)
* [arcee-ai/Llama-3.1-SuperNova-Lite](https://huggingface.co/arcee-ai/Llama-3.1-SuperNova-Lite)
* [v000000/L3.1-8B-RP-Test-003-Task_Arithmetic](https://huggingface.co/v000000/L3.1-8B-RP-Test-003-Task_Arithmetic)
* [Sao10K/L3.1-8B-Niitama-v1.1](https://huggingface.co/Sao10K/L3.1-8B-Niitama-v1.1) + [grimjim/Llama-3-Instruct-abliteration-LoRA-8B](https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-LoRA-8B)
* [v000000/L3.1-8B-RP-Test-002-Task_Arithmetic](https://huggingface.co/v000000/L3.1-8B-RP-Test-002-Task_Arithmetic) + [grimjim/Llama-3-Instruct-abliteration-LoRA-8B](https://huggingface.co/grimjim/Llama-3-Instruct-abliteration-LoRA-8B)

### Recipe

The following YAML configuration was used to produce this model:

```yaml
#Step1 - Add smarts to Niitama with alchemonaut's algorithm.

slices:
  - sources:
      - model: Sao10K/L3.1-8B-Niitama-v1.1+grimjim/Llama-3-Instruct-abliteration-LoRA-8B
        layer_range: [0, 32]
      - model: akjindal53244/Llama-3.1-Storm-8B
        layer_range: [0, 32]
merge_method: nearswap
base_model: Sao10K/L3.1-8B-Niitama-v1.1+grimjim/Llama-3-Instruct-abliteration-LoRA-8B
parameters:
  t:
    - value: 0.0001
dtype: bfloat16
out_type: float16

#Step 2 - Learn vectors onto Supernova 0.4(Niitorm)

models:
  - model: arcee-ai/Llama-3.1-SuperNova-Lite
    parameters:
      weight: 1.0
  - model: v000000/L3.1-Niitorm-8B-t0.0001
    parameters:
      weight: 0.4
merge_method: task_arithmetic
base_model: arcee-ai/Llama-3.1-SuperNova-Lite
parameters:
    normalize: false
dtype: float16

#Step 3 - Fully learn vectors onto Supernova 1.25(Niitorm)

models:
  - model: arcee-ai/Llama-3.1-SuperNova-Lite
    parameters:
      weight: 0.0
  - model: v000000/L3.1-Niitorm-8B-t0.0001
    parameters:
      weight: 1.25
merge_method: task_arithmetic
base_model: arcee-ai/Llama-3.1-SuperNova-Lite
parameters:
    normalize: false
dtype: float16

#Step 4 - Merge checkpoints and keep output/input Supernova heavy
#Merge with a triangular slerp from sophosympatheia.

models:
  - model: v000000/L3.1-8B-RP-Test-003-Task_Arithmetic
merge_method: slerp
base_model: v000000/L3.1-8B-RP-Test-002-Task_Arithmetic+grimjim/Llama-3-Instruct-abliteration-LoRA-8B
# This model needed some abliteration^
parameters:
  t:
    - value: [0, 0, 0.3, 0.4, 0.5, 0.6, 0.5, 0.4, 0.3, 0, 0]
dtype: float16

```

*SLERP distribution* used to smoothly blend the mostly Supernova base with the roleplay vectors:

![image/png](https://cdn-uploads.huggingface.co/production/uploads/64f74b6e6389380c77562762/GP2LMRvMkhVJwNDSEC4oU.png)


# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_v000000__L3.1-Storniitova-8B)

|      Metric       |Value|
|-------------------|----:|
|Avg.               |28.06|
|IFEval (0-Shot)    |78.17|
|BBH (3-Shot)       |30.81|
|MATH Lvl 5 (4-Shot)|13.29|
|GPQA (0-shot)      | 5.26|
|MuSR (0-shot)      | 9.96|
|MMLU-PRO (5-shot)  |30.84|