question model "bin files are updated to safetensors Add chat_template"
so you doo that for JANUS
following models dont can convert to gguf by llama.cpp
(tiny original)
https://huggingface.co/deepseek-ai/deepseek-vl2-tiny
and
(small original)
https://huggingface.co/deepseek-ai/deepseek-vl2-small/tree/main
also v3
https://huggingface.co/mlx-community/DeepSeek-V3-4bit/tree/main
you can take a look ?
I'm sorry, I'm not familiar with the gguf format. My method of conversion is very simple, just use this script.
https://github.com/Silver267/pytorch-to-safetensor-converter/blob/main/convert_to_safetensor.py
After that, I use the library safetensors
to partially extract the language model.
Compared to the other two models, DeepSeek-V3-4bit
is different, this one has been quantified by mlx
. It seems that it cannot be easily converted to other formats, and it is best to run only under the mlx
framework.
i see...
but seems your janus conversion can create at least a safaetensor
can you doo the same on the models above?
Yes, but I found that the deepseek-vl2-small
and deepseek-vl2-tiny
are themselves in safaetensor format.
Do you want to extract the language model part of it?
to answer, yes mlx is allready special a MAC(Apple) format.
extract the language model, yes but only if its not to hard to get ... ;)
i dont know much about - maybe ater extraction its usefull for LLM
GGUF is the most used format for low level users at home for LLM they can be converted by llama.cpp from pytorch. but only a hand full of formats are supported.
so you can easy use JAN or GPT4all to talk with these models.
So far, I'm not really sure what your ultimate needs are.
Want a janus gguf format?
Or the deepseek-vl2-tiny and deepseek-vl2-small gguf formats?
gguf
is something I'm not good at, and for the janus
architecture, additional code is needed to support inference. I won't be working on gguf
for a while.
i see... all fine
only extract the language model part of deep seek, tiny and small. can be that both the same and only the vision part differ ...