Alibaba-NLP

/

gte-large-en-v1.5

like 172

Follow

Alibaba-NLP 94

When using this model in a Triton ensemble via ONNX, what is the recommended "max_length" argument value of a string input into the tokenizer? I know that if that number doesn't correspond to the max_length the model was trained on, it can cause discrepancies. This is my current setup:

        tokens = self.tokenizer(
            [
                word.decode()
                for word in pb_utils.get_input_tensor_by_name(
                    requests[0], self.inputs[0]
                ).as_numpy()
            ],
            padding=True,
            truncation=True,
            return_token_type_ids=True,
            return_tensors="pt",
            max_length=8192,
        )