[QuestionGeneration] RuntimeError: Integer division of tensors using div or / is no longer supported

Environment info

• transformers version: 4.12.0
• Platform: Linux-5.4.0-88-generic-x86_64-with-debian-buster-sid
• Python version: 3.7.10
• PyTorch version (GPU?): 1.6.0 (True)
• Tensorflow version (GPU?): 2.6.0 (False)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: No
• Using distributed or parallel set-up in script?: No

Who can help

Models:

• T5ForConditionalGeneration : @patrickvonplaten, @patil-suraj

Information

Model I am using T5conditionalGeneration through Questgen.ai

The problem arises when using:

• my own modified scripts: (give details below)

from transformers import AutoTokenizer, AutoModelWithLMHead

def predict(sentence):
    tokenizer = AutoTokenizer.from_pretrained("flexudy/t5-base-multi-sentence-doctor")

    model = AutoModelWithLMHead.from_pretrained("flexudy/t5-base-multi-sentence-doctor")

    input_text = f"repair_sentence: {sentence}</s>"

    input_ids = tokenizer.encode(input_text, return_tensors="pt")

    outputs = model.generate(input_ids, max_length=32, num_beams=1)

    sentence = tokenizer.decode(outputs[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)

    return sentence

The tasks I am working on is:

• my own task :

  File "apis/text/text/boolean-question-generations/questgen/questgen.py", line 12, in predict
    output = qe.predict_boolq(payload)
  File "/opt/conda/lib/python3.7/site-packages/Questgen/main.py", line 238, in predict_boolq
    output = beam_search_decoding (input_ids, attention_masks,self.model,self.tokenizer)
  File "/opt/conda/lib/python3.7/site-packages/Questgen/encoding/encoding.py", line 18, in beam_search_decoding
    early_stopping=True
  File "/opt/conda/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/transformers/generation_utils.py", line 1064, in generate
    **model_kwargs,
  File "/opt/conda/lib/python3.7/site-packages/transformers/generation_utils.py", line 1839, in beam_search
    next_indices = (next_tokens / vocab_size).long()
RuntimeError: Integer division of tensors using div or / is no longer supported, and in a future release div will perform true division as in Python 3. Use true_divide or floor_divide (// in Python) instead.

To reproduce

Steps to reproduce the behavior:

pip install git+https://github.com/ramsrigouthamg/Questgen.ai
pip install git+https://github.com/boudinfl/pke.git

python -m nltk.downloader universal_tagset
python -m spacy download en 

from transformers import AutoTokenizer, AutoModelWithLMHead

def predict(sentence):
    tokenizer = AutoTokenizer.from_pretrained("flexudy/t5-base-multi-sentence-doctor")

    model = AutoModelWithLMHead.from_pretrained("flexudy/t5-base-multi-sentence-doctor")

    input_text = f"repair_sentence: {sentence}</s>"

    input_ids = tokenizer.encode(input_text, return_tensors="pt")

    outputs = model.generate(input_ids, max_length=32, num_beams=1)

    sentence = tokenizer.decode(outputs[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)

    return sentence

Expected behavior

No error

Fix found

https://github.com/huggingface/transformers/blob/master/src/transformers/generation_utils.py#L1839

use // instead of /

[mwojnars]

@patrickvonplaten Thanks. It seems to be working fine now.

[JeetRoy97]

I am still getting the error in generation_utils.py. The issue is not yet resolved in Transformers 4.15.0