Bug on the tokenizer, using the code that you provided for the inference.
I force downloaded the tokenizer many times in case of failure (or file corruption) but it doesnt seem to be the case.
Firstly i had to change the config file (An error occured reuqired only 2 arguments type and factor) :
"rope_scaling": {
"factor": 8.0,
"high_freq_factor": 4.0,
"low_freq_factor": 1.0,
"original_max_position_embeddings": 8192,
"rope_type": "llama3"
},
Second:
Even when i change it there seems to be another error with the tokenizer file :
File ~/.local/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py:111, in PreTrainedTokenizerFast.init(self, *args, **kwargs)
108 fast_tokenizer = copy.deepcopy(tokenizer_object)
109 elif fast_tokenizer_file is not None and not from_slow:
110 # We have a serialization from tokenizers which let us directly build the backend
--> 111 fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
112 elif slow_tokenizer is not None:
113 # We need to convert a slow tokenizer to build the backend
114 fast_tokenizer = convert_slow_tokenizer(slow_tokenizer)
Exception: data did not match any variant of untagged enum ModelWrapper at line 1356219 column 3
Could you please assist me? I would like to test this LLM.
What transformers version are you using? after transformers == 4.45.0 the way the tokenizers are handled, changed.
I'm guessing you're trying to load the model with a transformers version < 4.45.0 which would explain both the RoPE config error and the tokenizer loading error
File upload does not work, it would be nice to set it up to make some rag testing
You can create one by yourself
@Ptrnk
gguf from hg
https://github.com/ggerganov/llama.cpp/discussions/2948
gguf to ollama
https://github.com/ollama/ollama/blob/main/docs/import.md#importing-a-gguf-based-model-or-adapter
Although I believe you dont need ollama at all :P
File upload does not work, it would be nice to set it up to make some rag testing
We are working to integrate file upload and tool calling to our UI, it's not our top priority at the moment but we'll get to it and finalize it soon! Thank you for your interest in the model!