Love the idea
#1
by
flashvenom
- opened
I love the idea, Airoboros x SuperHOT seems like the perfect cross-over given Airoboros training data and the fact that a lot of it is >2048 tokens, does this need any special monkey patch etc. to run?
https://huggingface.co/flashvenom/Airoboros-13B-SuperHOT-8K-GPTQ quantized this, btw which version of Airoboros and SuperHOT did you use? Mind sharing the code you used to merge the models?
Awesome! Thanks! I used Airboros 1.2-gpt4 i believe, and the latest SuperHOT 8k non-rlhf LoRA kaiokendev uploaded. I’ll add the model card later, sorry about that.
I’ll add the code to the card.
Hey, model card created.
The model card says Airoboros 1.4, is that accurate? Thanks for adding it btw