2bit

by KnutJaegersberg - opened Oct 8, 2023

Oct 8, 2023

You should get your model 2 bit quantized by https://huggingface.co/GreenBitAI/LLaMA-3B-2bit-groupsize32
so we can use all as much as possible context length in best quality on consumer hardware.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment