File size: 3,501 Bytes
97e75ab da0a52b 97e75ab |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 |
---
base_model:
- mistralai/Mixtral-8x7B-v0.1
- mistralai/Mixtral-8x7B-v0.1
- Doctor-Shotgun/limarp-zloss-mixtral-8x7b-qlora
- KoboldAI/Mixtral-8x7B-Holodeck-v1
- jondurbin/bagel-dpo-8x7b-v0.2
- mistralai/Mixtral-8x7B-Instruct-v0.1
tags:
- mergekit
- merge
license: apache-2.0
---
I have no idea what I’m doing… if this causes the apocalypse someone please let me know.
DonutHole-8x7B 8.0bpw h8 EXL2
Includes [measurement.json](https://huggingface.co/FuturisticVibes/DonutHole-8x7B-8.0bpw-h8-exl2/tree/measurement) file for further quantization
Original Model: https://huggingface.co/ycros/DonutHole-8x7B
# Original Model Card
# DonutHole-8x7B
[GGUF versions here](https://huggingface.co/ycros/DonutHole-8x7B-GGUF)
Bagel, Mixtral Instruct, Holodeck, LimaRP.
> What mysteries lie in the hole of a donut?
Good with Alpaca prompt formats, also works with Mistral format. See usage details below.
This is similar to [BagelMIsteryTour](https://huggingface.co/ycros/BagelMIsteryTour-v2-8x7B), but I've swapped out Sensualize for the new Holodeck.
I'm not sure if it's better or not yet, or how it does at higher (8k+) contexts just yet.
Similar sampler advice applies as for BMT: minP (0.07 - 0.3 to taste) -> temp (either dynatemp 0-4ish, or like a temp of 3-4 with a smoothing factor of around 2.5ish).
And yes, that's temp last. It does okay without rep pen up to a point, it doesn't seem to get into a complete jam, but it can start to repeat sentences,
so you'll probably need some, perhaps 1.02-1.05 at a 1024 range seems okayish.
(rep pen sucks, but there are better things coming).
I've mainly tested with LimaRP style Alpaca prompts (instruction/input/response), and briefly with Mistral's own format.
**Full credit to all the model and dataset authors, I am but a derp with compute and a yaml file.**
---
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) as a base.
### Models Merged
The following models were included in the merge:
* [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) + [Doctor-Shotgun/limarp-zloss-mixtral-8x7b-qlora](https://huggingface.co/Doctor-Shotgun/limarp-zloss-mixtral-8x7b-qlora)
* [KoboldAI/Mixtral-8x7B-Holodeck-v1](https://huggingface.co/KoboldAI/Mixtral-8x7B-Holodeck-v1)
* [jondurbin/bagel-dpo-8x7b-v0.2](https://huggingface.co/jondurbin/bagel-dpo-8x7b-v0.2)
* [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
### Configuration
The following YAML configuration was used to produce this model:
```yaml
base_model: mistralai/Mixtral-8x7B-v0.1
models:
- model: mistralai/Mixtral-8x7B-v0.1+Doctor-Shotgun/limarp-zloss-mixtral-8x7b-qlora
parameters:
density: 0.5
weight: 0.2
- model: KoboldAI/Mixtral-8x7B-Holodeck-v1
parameters:
density: 0.5
weight: 0.2
- model: mistralai/Mixtral-8x7B-Instruct-v0.1
parameters:
density: 0.6
weight: 1.0
- model: jondurbin/bagel-dpo-8x7b-v0.2
parameters:
density: 0.6
weight: 0.5
merge_method: dare_ties
dtype: bfloat16
``` |