File size: 617 Bytes
de263a7 60ecc08 de263a7 94fa480 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
---
license: mit
language:
- en
pipeline_tag: text-generation
inference: false
tags:
- dare
- super mario merge
- pytorch
- mixtral
- merge
---
# mixtral dare test
The following were merged with DARE using [https://github.com/martyn/safetensors-merge-supermario](https://github.com/martyn/safetensors-merge-supermario)
## Mergelist
```
mistralai/Mixtral-8x7B-Instruct-v0.1
Open-Orca/Mixtral-SlimOrca-8x7B
```
## Merge command
```
python3 hf_merge.py to_merge_mixtral0.txt mixtral-0 -p 0.3 -lambda 2.1
```
## Notes
* This is primarily a test to see if merging mixtral models works.
* MoE gates are not merged. |