8.0bpw-h8-exl2 quant of this model
@LoneStriker Thanks for all your efforts. This model feels really great, could you please make a 8.0bpw-h8-exl2 quant of it, as for the rest of your models?
Agreed that the model is really good. Many of the newer 2x34B merges have been surprisingly good. I can add 8.0 to the list of quants, but very few people have more than 48 GB VRAM to load them.
Thanks again. I got 4 hard-modded 2080Ti 22G so I could explore a wider range of (quantized) models but 88G still falls short for most large full models. They are also surpricesingly cheap (~350$ each).
8.0bpw quant up:
https://huggingface.co/LoneStriker/Mixtral_34Bx2_MoE_60B-8.0bpw-h8-exl2
Have not heard of modded 2080 Ti cards, interesting. Used 3090s here are around $700, so I have some of those instead for more VRAM.
Thanks again. I got 4 hard-modded 2080Ti 22G so I could explore a wider range of (quantized) models but 88G still falls short for most large full models. They are also surpricesingly cheap (~350$ each).
where and how to buy these cheap hard-modded 2080Ti ?
thanks
Thanks again. I got 4 hard-modded 2080Ti 22G so I could explore a wider range of (quantized) models but 88G still falls short for most large full models. They are also surpricesingly cheap (~350$ each).
where and how to buy these cheap hard-modded 2080Ti ?
thanks
I live in China, so I got them on 闲鱼(that's taobao for used items, just search 2080ti 22G, it's easy to find ones around 2400 CNY).
Thanks again. I got 4 hard-modded 2080Ti 22G so I could explore a wider range of (quantized) models but 88G still falls short for most large full models. They are also surpricesingly cheap (~350$ each).
where and how to buy these cheap hard-modded 2080Ti ?
thanksI live in China, so I got them on 闲鱼(that's taobao for used items, just search 2080ti 22G, it's easy to find ones around 2400 CNY).
我在pdd看到了,私聊qq 206887187,请教一下怎么配置