wnma3mz

/

Janus-Pro-7B-LM

like 1

I'm sorry, I'm not familiar with the gguf format. My method of conversion is very simple, just use this script.

https://github.com/Silver267/pytorch-to-safetensor-converter/blob/main/convert_to_safetensor.py

After that, I use the library safetensors to partially extract the language model.

Compared to the other two models, DeepSeek-V3-4bit is different, this one has been quantified by mlx. It seems that it cannot be easily converted to other formats, and it is best to run only under the mlx framework.

i see...
but seems your janus conversion can create at least a safaetensor

can you doo the same on the models above?

Yes, but I found that the deepseek-vl2-small and deepseek-vl2-tiny are themselves in safaetensor format.

Do you want to extract the language model part of it?

to answer, yes mlx is allready special a MAC(Apple) format.

extract the language model, yes but only if its not to hard to get ... ;)
i dont know much about - maybe ater extraction its usefull for LLM

GGUF is the most used format for low level users at home for LLM they can be converted by llama.cpp from pytorch. but only a hand full of formats are supported.
so you can easy use JAN or GPT4all to talk with these models.

So far, I'm not really sure what your ultimate needs are.

• Want a janus gguf format?

• Or the deepseek-vl2-tiny and deepseek-vl2-small gguf formats?

gguf is something I'm not good at, and for the janus architecture, additional code is needed to support inference. I won't be working on gguf for a while.

i see... all fine

only extract the language model part of deep seek, tiny and small. can be that both the same and only the vision part differ ...