Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error: ``Cannot colocate nodes...''

See original GitHub issue

Hi Wenxuan,

Many thanks for this wonderful and educational implementation. I have encountered an error while trying python config.py --mode debug.

InvalidArgumentError (see above for traceback): Cannot colocate nodes ‘global_norm/L2Loss_32’ and ‘gradients/match/bw/CudnnRNN_grad/CudnnRNNBackprop’ because no device type supports both of those nodes and the other nodes colocated with them. Colocation Debug Info: Colocation group had the following types and devices: CudnnRNNBackprop: GPU Identity: L2Loss: CPU [[Node: global_norm/L2Loss_32 = L2LossT=DT_FLOAT, _class=[“loc:@gradients/match/bw/CudnnRNN_grad/CudnnRNNBackprop”]]]

All the configs are by default.

Issue Analytics

State:
Created 6 years ago
Comments:9 (3 by maintainers)

Top GitHub Comments

1reaction

xiongyifancommented, Jan 3, 2018

@yuchenlin … … I am also confusing. Maybe because of the version. When I was running a RNN example that uses the CudnnLSTM, I could not use Adam to optimize the CudnnLSTM. But in this project, we can use Adam to optimize CudnnGRU. I am not sure … I am confusing …

0reactions

mickyyu96commented, Jun 25, 2018

@yuchenlin how you solve the problem ? i got the same issue

Top Results From Across the Web

Cannot colocate nodes, Cannot merge devices with ... - GitHub

I created a keras_model_fn and am trying to train the model on 3 c4 instances. Unfortunately, I get the following error (detailed below)....

Distributed tensorflow cannot colocate nodes /job:master/task ...

InvalidArgumentError (see above for traceback): Cannot colocate nodes 'Decoder-Word-Embedding/embeddings' and ...

how to fix this error when trainin… | Apple Developer Forums

Error is here InvalidArgumentError: Cannot assign a device for operation ... was colocated with a group of nodes that required incompatible device ...

Assigning Pods to Nodes - Kubernetes

You can constrain a Pod so that it is restricted to run on particular node(s), or to prefer to run on particular nodes....

Chapter 17. Creating cluster resources that are active on ...

The following command creates a colocation constraint to ensure that the resource webfarm-stats runs on the same node as an active copy of...