GGUF and llama.cpp

by skruse - opened Nov 13

Nov 13

I tried to test this model with llama.cpp and the quants from "DevQuasar/VAGOsolutions.SauerkrautLM-v2-14b-DPO-GGUF" and it only output random timestamps.

I tried with and without flash_attention, both times similiar result.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment