Following the instructions
Hi,
I took the Arcee-Blitz-Q4_K_M.gguf and tried it in LM Studio on a 16GB Nvidia Quadro P5000, but it wouldn't start at all. Then I put it in to Jan and tried to translate the subtitles for a popular science film, by specifying the system and user promptes. As a result, I received about 4 tokens per second, but instead of a translation, he first gave a description. On the next attempt, he gave a resume. Then I managed to get him to start translating, but he lost the numbering. It is worth saying that the original model does not cope well with this task. What kind of model would you recommend for such work, taking into account your knowledge of scientific terminology and support for European languages?
Try virtuoso-lite or virtuoso-small, both should be in our repo. I can't speak to the GGUFs because the tokenizer might be a bit strange after the surgery we do. I can try replacing with mistrals stock tokenizer and see.