teknium

/

airoboros-mistral2.2-7b

like 36

Hi there! I am quite new to converting models, so please excuse me if i'm doing something totally wrong in the first place.
I want to run this airoboros-trained version using ollama.ai/llama.cpp which requires a gguf version as it seems.

So far, I've downloaded the model and tried to start the conversion process like so:

python llama.cpp/convert.py airoboros-mistral2.2-7b --outfile airoboros-mistral2.2-7b.gguf

However, I am getting the following error back:

Loading model file airoboros-mistral2.2-7b\pytorch_model-00001-of-00002.bin
Loading model file airoboros-mistral2.2-7b\pytorch_model-00001-of-00002.bin
Loading model file airoboros-mistral2.2-7b\pytorch_model-00002-of-00002.bin
params = Params(n_vocab=32000, n_embd=4096, n_layer=32, n_ctx=32768, n_ff=14336, n_head=32, n_head_kv=8, f_norm_eps=1e-05, f_rope_freq_base=10000.0, f_rope_scale=None, ftype=None, path_model=WindowsPath('airoboros-mistral2.2-7b'))
Loading vocab file 'airoboros-mistral2.2-7b\tokenizer.model', type 'spm'
Traceback (most recent call last):
  File "C:\Users\spaceemotion\code\gguf\llama.cpp\convert.py", line 1193, in <module>
    main()
  File "C:\Users\spaceemotion\code\gguf\llama.cpp\convert.py", line 1175, in main
    vocab = load_vocab(vocab_dir, args.vocabtype)
  File "C:\Users\spaceemotion\code\gguf\llama.cpp\convert.py", line 1086, in load_vocab
    return SentencePieceVocab(path, added_tokens_path if added_tokens_path.exists() else None)
  File "C:\Users\spaceemotion\code\gguf\llama.cpp\convert.py", line 372, in __init__
    raise Exception(f"Expected added token IDs to be sequential and start at {len(added_tokens)}; got {actual_ids}")
Exception: Expected added token IDs to be sequential and start at 3; got [0, 1, 2]

Is this something because of my setup or how can I go about fixing this? Thanks in advance!