Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Device error on TokenClassificationPipeline

See original GitHub issue

Environment info

transformers version: 4.11.0
Platform: Linux-5.14.8-arch1-1-x86_64-with-arch
Python version: 3.7.11
PyTorch version (GPU?): 1.9.1+cu102 (True)
Tensorflow version (GPU?): not installed (NA)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: True
Using distributed or parallel set-up in script?: False

Who can help

Library:

pipelines: @LysandreJik

Information

Model I am using (Bert, XLNet …):

The problem arises when using:

the official example scripts: (give details below)

The tasks I am working on is:

an official GLUE/SQUaD task: (give the name)

To reproduce

Steps to reproduce the behavior:

Create a pipe = TokenClassificationPipeline(model=DistilBertForTokenClassification.from_pretrained("PATH"))
Pipe some text in pipe(["My", "text", "tokens"])
Get a TypeError: can't convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to host memory first.

Expected behavior

Be able to run the pipeline

The pipeline should bring data to gpu/cpu or model to gpu/cpu and vice versa.

The traceback

In .venv/lib/python3.7/site-packages/transformers/pipelines/token_classification.py:209 in _forward                                                                                
    206 │   │   if self.framework == "tf":                                                         
    207 │   │   │   outputs = self.model(model_inputs.data)[0][0].numpy()                          
    208 │   │   else:                                                                              
 ❱ 209 │   │   │   outputs = self.model(**model_inputs)[0][0].numpy()   <== HERE
    210 │   │   return {                                                                           
    211 │   │   │   "outputs": outputs,                                                            
    212 │   │   │   "special_tokens_mask": special_tokens_mask,

Placing a .cpu() would solve the problem

Thanks in advance for any help Have a wonderful day

Issue Analytics

State:
Created 2 years ago
Comments:6 (5 by maintainers)

Top GitHub Comments

2reactions

LysandreJikcommented, Sep 30, 2021

Nice catch! Would you like to open a PR with the fix?

1reaction

mallorbccommented, Sep 30, 2021

similar issue later in the file, line 223

 220 │   │   sentence = model_outputs["sentence"]                           │
│   221 │   │   input_ids = model_outputs["input_ids"][0]                      │
│   222 │   │   offset_mapping = model_outputs["offset_mapping"][0] if model_o │
│ ❱ 223 │   │   special_tokens_mask = model_outputs["special_tokens_mask"][0].numpy() │
│   224 │   │                                                                  │
│   225 │   │   scores = np.exp(outputs) / np.exp(outputs).sum(-1, keepdims=Tr │
│   226 │   │   pre_entities = self.gather_pre_entities(