Error in export_2hlayers_model.py InvalidArgumentError: Cannot assign a device for operation private-input/encode/mul
See original GitHub issueI ran the export_2hlayers_model.py example and got this error. I ran it on a Macbook pro CPU.
Complete error goes like
File "<ipython-input-24-adaa386ad418>", line 1, in <module>
runfile('/Users/hrishikesh/Hrishikesh/Projects/tf-encrypted/examples/export_2hlayers_model.py', wdir='/Users/hrishikesh/Hrishikesh/Projects/tf-encrypted/examples')
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile
execfile(filename, namespace)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "/Users/hrishikesh/Hrishikesh/Projects/tf-encrypted/examples/export_2hlayers_model.py", line 47, in <module>
sess.run(tf.global_variables_initializer())
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 929, in run
run_metadata_ptr)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
run_metadata)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
InvalidArgumentError: Cannot assign a device for operation private-input/encode/mul: Operation was explicitly assigned to /job:localhost/replica:0/task:0/device:CPU:3 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.
[[node private-input/encode/mul (defined at /Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py:530) = Mul[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:3"](private-input/Const, private-input/encode/mul/y)]]
Caused by op 'private-input/encode/mul', defined at:
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/ipython/start_kernel.py", line 245, in <module>
main()
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/ipython/start_kernel.py", line 241, in main
kernel.start()
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 477, in start
ioloop.IOLoop.instance().start()
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 177, in start
super(ZMQIOLoop, self).start()
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tornado/ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
self._handle_recv()
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
self._run_callback(callback, msg)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 235, in dispatch_shell
handler(stream, idents, msg)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 196, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 533, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2698, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2808, in run_ast_nodes
if self.run_code(code, result):
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2862, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-10-84f822066520>", line 1, in <module>
runfile('/Users/hrishikesh/Hrishikesh/Projects/Private Machine Learning/tfencrypted/Get_Dataset.py', wdir='/Users/hrishikesh/Hrishikesh/Projects/Private Machine Learning/tfencrypted')
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile
execfile(filename, namespace)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "/Users/hrishikesh/Hrishikesh/Projects/Private Machine Learning/tfencrypted/Get_Dataset.py", line 4, in <module>
from convert import encode
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/Users/hrishikesh/Hrishikesh/Projects/Private Machine Learning/tfencrypted/convert.py", line 62, in <module>
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/convert/convert.py", line 56, in convert
x = self.protocol.define_private_input(input_player, item)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py", line 452, in define_private_input
output = helper(v)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py", line 431, in helper
w = self._encode(v, apply_scaling)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py", line 530, in _encode
scaled = rationals * self.fixedpoint_config.scaling_factor
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 878, in binary_op_wrapper
return func(x, y, name=name)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1131, in _mul_dispatch
return gen_math_ops.mul(x, y, name=name)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 5042, in mul
"Mul", x=x, y=y, name=name)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
op_def=op_def)
File "/Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1770, in __init__
self._traceback = tf_stack.extract_stack()
InvalidArgumentError (see above for traceback): Cannot assign a device for operation private-input/encode/mul: Operation was explicitly assigned to /job:localhost/replica:0/task:0/device:CPU:3 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.
[[node private-input/encode/mul (defined at /Users/hrishikesh/anaconda3/envs/tfencrypt/lib/python3.6/site-packages/tf_encrypted/protocol/pond.py:530) = Mul[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:3"](private-input/Const, private-input/encode/mul/y)]]
Issue Analytics
- State:
- Created 5 years ago
- Comments:10 (10 by maintainers)
Top Results From Across the Web
Cannot assign a device for operation embedding_1 ... - GitHub
When I call model.fit_generator(...) I get an error: InvalidArgumentError: Cannot assign a device for operation embedding/embeddings/Initializer ...
Read more >Cannot assign a device for operation ReadVariableOp
I am getting the following error which I assume is the same issue as above. Colocation Debug Info: Colocation group had the following...
Read more >Tensorflow-gpu Error: InvalidArgumentError: Cannot assign a ...
You're trying to multiple tensors with a tf.int32 ( DT_INT32 ) datatype on the GPU. The error message is saying that there is...
Read more >InvalidArgumentError: Cannot assign a device for operation
I encountered an error while executing the code on the site below. keras-io/image_captioning.py at master · keras-team/keras-io · GitHub And ...
Read more >GPU acceleration in WSL - FAQ - Microsoft Learn
Why am I getting device assignment or node colocation errors? ... TensorFlow operations will automatically be assigned to the DirectML device if possible....
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I got this issue resolved. Not really sure why and how since all I did was remove my conda environment and create it again with tensorflow 1.12
Closing up; second problem was addresses and fixed in https://github.com/tf-encrypted/tf-encrypted/issues/457.