Add support for truncation argument when calling a Pipeline

🚀 Feature request

Currently, only the padding argument is supported when calling a pipeline, and it’s not possible to pass truncation argument. For example, running the following code sample would raise an error:

import transformers as trf

model = trf.pipeline(task='feature-extraction', model='bert-base-cased')
output = model('a sample text', padding=False, truncation=True)

Motivation

If toggling padding is supported, then why truncation shouldn’t be?

Your contribution

I think to achieve this, same as padding, only a truncation argument should be added to _parse_and_tokenize method and also when calling the tokenizer. If that’s the case, I would be willing to work on a PR.

[buhrmann]

Hi, even though this has been closed as stale, without comment or supposed fix, it seems that in recent versions you can in fact pass both truncation and padding arguments to the pipeline’s __call__ method, and it will correctly use them when tokenizing. I’ve tested it with long texts that fail without the truncation argument, and it seems to work as expected.

[alexbw]

+1 on this