question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Error in export_2hlayers_model.py InvalidArgumentError: Cannot assign a device for operation private-input/encode/mul

See original GitHub issue

I ran the export_2hlayers_model.py example and got this error. I ran it on a Macbook pro CPU.

Complete error goes like

File "<ipython-input-24-adaa386ad418>", line 1, in <module>
   runfile('/Users/hrishikesh/Hrishikesh/Projects/tf-encrypted/examples/export_2hlayers_model.py', wdir='/Users/hrishikesh/Hrishikesh/Projects/tf-encrypted/examples')

 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile
   execfile(filename, namespace)

 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile
   exec(compile(f.read(), filename, 'exec'), namespace)

 File "/Users/hrishikesh/Hrishikesh/Projects/tf-encrypted/examples/export_2hlayers_model.py", line 47, in <module>
   sess.run(tf.global_variables_initializer())

 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
   run_metadata_ptr)

 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
   feed_dict_tensor, options, run_metadata)

 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
   run_metadata)

 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
   raise type(e)(node_def, op, message)

InvalidArgumentError: Cannot assign a device for operation private-input/encode/mul: Operation was explicitly assigned to /job:localhost/replica:0/task:0/device:CPU:3 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.
    [[node private-input/encode/mul (defined at /Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py:530)  = Mul[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:3"](private-input/Const, private-input/encode/mul/y)]]

Caused by op 'private-input/encode/mul', defined at:
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/ipython/start_kernel.py", line 245, in <module>
   main()
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/ipython/start_kernel.py", line 241, in main
   kernel.start()
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 477, in start
   ioloop.IOLoop.instance().start()
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 177, in start
   super(ZMQIOLoop, self).start()
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tornado/ioloop.py", line 888, in start
   handler_func(fd_obj, events)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
   return fn(*args, **kwargs)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
   self._handle_recv()
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
   self._run_callback(callback, msg)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
   callback(*args, **kwargs)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
   return fn(*args, **kwargs)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
   return self.dispatch_shell(stream, msg)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 235, in dispatch_shell
   handler(stream, idents, msg)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
   user_expressions, allow_stdin)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 196, in do_execute
   res = shell.run_cell(code, store_history=store_history, silent=silent)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 533, in run_cell
   return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2698, in run_cell
   interactivity=interactivity, compiler=compiler, result=result)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2808, in run_ast_nodes
   if self.run_code(code, result):
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2862, in run_code
   exec(code_obj, self.user_global_ns, self.user_ns)
 File "<ipython-input-10-84f822066520>", line 1, in <module>
   runfile('/Users/hrishikesh/Hrishikesh/Projects/Private Machine Learning/tfencrypted/Get_Dataset.py', wdir='/Users/hrishikesh/Hrishikesh/Projects/Private Machine Learning/tfencrypted')
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile
   execfile(filename, namespace)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile
   exec(compile(f.read(), filename, 'exec'), namespace)
 File "/Users/hrishikesh/Hrishikesh/Projects/Private Machine Learning/tfencrypted/Get_Dataset.py", line 4, in <module>
   from convert import encode
 File "<frozen importlib._bootstrap>", line 971, in _find_and_load
 File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
 File "<frozen importlib._bootstrap_external>", line 678, in exec_module
 File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
 File "/Users/hrishikesh/Hrishikesh/Projects/Private Machine Learning/tfencrypted/convert.py", line 62, in <module>
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/convert/convert.py", line 56, in convert
   x = self.protocol.define_private_input(input_player, item)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py", line 452, in define_private_input
   output = helper(v)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py", line 431, in helper
   w = self._encode(v, apply_scaling)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py", line 530, in _encode
   scaled = rationals * self.fixedpoint_config.scaling_factor
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 878, in binary_op_wrapper
   return func(x, y, name=name)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1131, in _mul_dispatch
   return gen_math_ops.mul(x, y, name=name)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 5042, in mul
   "Mul", x=x, y=y, name=name)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
   op_def=op_def)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
   return func(*args, **kwargs)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
   op_def=op_def)
 File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
   self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Cannot assign a device for operation private-input/encode/mul: Operation was explicitly assigned to /job:localhost/replica:0/task:0/device:CPU:3 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.
    [[node private-input/encode/mul (defined at /Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py:530)  = Mul[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:3"](private-input/Const, private-input/encode/mul/y)]]

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:10 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
kamathhrishicommented, Feb 13, 2019

I got this issue resolved. Not really sure why and how since all I did was remove my conda environment and create it again with tensorflow 1.12

0reactions
mortendahlcommented, May 7, 2019

Closing up; second problem was addresses and fixed in https://github.com/tf-encrypted/tf-encrypted/issues/457.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Cannot assign a device for operation embedding_1 ... - GitHub
When I call model.fit_generator(...) I get an error: InvalidArgumentError: Cannot assign a device for operation embedding/embeddings/Initializer ...
Read more >
Cannot assign a device for operation ReadVariableOp
I am getting the following error which I assume is the same issue as above. Colocation Debug Info: Colocation group had the following...
Read more >
Tensorflow-gpu Error: InvalidArgumentError: Cannot assign a ...
You're trying to multiple tensors with a tf.int32 ( DT_INT32 ) datatype on the GPU. The error message is saying that there is...
Read more >
InvalidArgumentError: Cannot assign a device for operation
I encountered an error while executing the code on the site below. keras-io/image_captioning.py at master · keras-team/keras-io · GitHub And ...
Read more >
GPU acceleration in WSL - FAQ - Microsoft Learn
Why am I getting device assignment or node colocation errors? ... TensorFlow operations will automatically be assigned to the DirectML device if possible....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found