CUDA error for long sentences?

Hello, I am using Sentence Transformers, specifically distiluse-base-multilingual-cased, to compute semantic similarity for a list of sentences. I have been getting CUDA errors (last of which was CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)` ) on a specific dataset, and after some investigation I realized this was due to a specific sentence of 2115 characters, which gets tokenized to 580 tokens. I am aware this is longer than the model max_seq_length (which I set to 512), but I was expecting the sentence to be automatically truncated to the maximum length with no error arising. Am I doing something wrong? I am running version 0.3.2 (transformers version 3.0.2).

[nreimers]

Try

model._first_module().max_seq_length = 509

Transformer models add up to 3 special tokens like [CLS] and/or [SEP] to your input sentence. So if your sentence is 512 tokens, and 3 special tokens are added, this would result in too many tokens passed to the transformers model.

[thesby]

I fix it by modify the Transformer class

    def tokenize(self, text: Union[str, List[str]]) -> List[int]:
        """
        Tokenizes a text and maps tokens to token-ids
        """
        if isinstance(text, str):
            return self.tokenizer.convert_tokens_to_ids(self.tokenizer.tokenize(text, max_length=self.max_seq_length-3))
        else:
            return [self.tokenizer.convert_tokens_to_ids(self.tokenizer.tokenize(t, max_length=self.max_seq_length-3)) for t in text]