timpal0l commited on
Commit
4b704d7
1 Parent(s): a0b397f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -1
README.md CHANGED
@@ -17,4 +17,23 @@ pipeline_tag: text-generation
17
  ![](https://huggingface.co/timpal0l/flashbeagle-7b/resolve/main/model_img.png?download=true)
18
 
19
 
20
- This model is a [Slerp Merge](https://github.com/cg123/mergekit/blob/main/mergekit/merge_methods/slerp.py) of [timpal0l/Mistral-7B-v0.1-flashback-v2/](https://huggingface.co/timpal0l/Mistral-7B-v0.1-flashback-v2) and [RJuro/munin-neuralbeagle-7b](https://huggingface.co/RJuro/munin-neuralbeagle-7b).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
  ![](https://huggingface.co/timpal0l/flashbeagle-7b/resolve/main/model_img.png?download=true)
18
 
19
 
20
+ This model is a [Slerp Merge](https://github.com/cg123/mergekit/blob/main/mergekit/merge_methods/slerp.py) of [timpal0l/Mistral-7B-v0.1-flashback-v2/](https://huggingface.co/timpal0l/Mistral-7B-v0.1-flashback-v2) and [RJuro/munin-neuralbeagle-7b](https://huggingface.co/RJuro/munin-neuralbeagle-7b).
21
+ ## config.yaml
22
+ ```yaml
23
+ slices:
24
+ - sources:
25
+ - model: timpal0l/Mistral-7B-v0.1-flashback-v2
26
+ layer_range: [0, 32]
27
+ - model: RJuro/munin-neuralbeagle-7b
28
+ layer_range: [0, 32]
29
+ merge_method: slerp
30
+ base_model: timpal0l/Mistral-7B-v0.1-flashback-v2
31
+ parameters:
32
+ t:
33
+ - filter: self_attn
34
+ value: [0, 0.5, 0.3, 0.7, 1]
35
+ - filter: mlp
36
+ value: [1, 0.5, 0.7, 0.3, 0]
37
+ - value: 0.5
38
+ dtype: bfloat16
39
+ ```