Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

failed to get the ith layer output if Dropout is contained

See original GitHub issue

def get_ith_layer_output(model, X, i):
    ''' if the model contains Dropout,  it will fail '''
    get_ith_layer = keras.backend.function(
        [model.layers[0].input], [model.layers[i].output])
    layer_output = get_ith_layer([X])[0]
    return layer_output
    
N = nD.shape[1]    
model = Sequential(layers=[
    Dense(150, input_dim=N, init='normal', activation='relu'), # Dropout(0.4),
    Dense(50, init='normal'), Activation(PReLU()),  Dropout(0.2), 
    Dense(1, init='normal', activation='sigmoid'),
])

model.compile(loss='binary_crossentropy',
        optimizer='adadelta', # https://github.com/fchollet/keras/blob/master/keras/optimizers.py
        metrics=['accuracy',])

model.fit(nD.values, D.good_flag.values, batch_size=64, nb_epoch=5, class_weight={1:1,0:10}, validation_split=0.8)

out = {}
for i in range( len(model.layers) ):
    out[i] = get_ith_layer_output(model, nD.values, i)
    display(out[i].shape)

(10000, 150)
(10000, 50)
(10000, 50)
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1020     try:
-> 1021       return fn(*args)
   1022     except errors.OpError as e:

/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
   1002                                  feed_dict, fetch_list, target_list,
-> 1003                                  status, run_metadata)
   1004 

/opt/anaconda3/lib/python3.5/contextlib.py in __exit__(self, type, value, traceback)
     65             try:
---> 66                 next(self.gen)
     67             except StopIteration:

/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py in raise_exception_on_not_ok_status()
    468           compat.as_text(pywrap_tensorflow.TF_Message(status)),
--> 469           pywrap_tensorflow.TF_GetCode(status))
    470   finally:

InvalidArgumentError: You must feed a value for placeholder tensor 'keras_learning_phase' with dtype bool
	 [[Node: keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

During handling of the above exception, another exception occurred:

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-163-02d2bec78984> in <module>()
      3 out = {}
      4 for i in range( len(model.layers) ):
----> 5     out[i] = get_ith_layer_output(model, X, i)
      6     display(out[i].shape)

<ipython-input-162-b6552422d94b> in get_ith_layer_output(model, X, i)
      4     get_ith_layer = keras.backend.function(
      5         [model.layers[0].input], [model.layers[i].output])
----> 6     layer_output = get_ith_layer([X])[0]
      7     return layer_output
      8 

/opt/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py in __call__(self, inputs)
   1601         session = get_session()
   1602         updated = session.run(self.outputs + [self.updates_op],
-> 1603                               feed_dict=feed_dict)
   1604         return updated[:len(self.outputs)]
   1605 

/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
    764     try:
    765       result = self._run(None, fetches, feed_dict, options_ptr,
--> 766                          run_metadata_ptr)
    767       if run_metadata:
    768         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
    962     if final_fetches or final_targets:
    963       results = self._do_run(handle, final_targets, final_fetches,
--> 964                              feed_dict_string, options, run_metadata)
    965     else:
    966       results = []

/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
   1012     if handle is None:
   1013       return self._do_call(_run_fn, self._session, feed_dict, fetch_list,
-> 1014                            target_list, options, run_metadata)
   1015     else:
   1016       return self._do_call(_prun_fn, self._session, handle, feed_dict,

/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1032         except KeyError:
   1033           pass
-> 1034       raise type(e)(node_def, op, message)
   1035 
   1036   def _extend_graph(self):

InvalidArgumentError: You must feed a value for placeholder tensor 'keras_learning_phase' with dtype bool
	 [[Node: keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

Caused by op 'keras_learning_phase', defined at:
  File "/opt/anaconda3/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/anaconda3/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/anaconda3/lib/python3.5/site-packages/ipykernel/__main__.py", line 3, in <module>
    app.launch_new_instance()
  File "/opt/anaconda3/lib/python3.5/site-packages/traitlets/config/application.py", line 653, in launch_instance
    app.start()
  File "/opt/anaconda3/lib/python3.5/site-packages/ipykernel/kernelapp.py", line 474, in start
    ioloop.IOLoop.instance().start()
  File "/opt/anaconda3/lib/python3.5/site-packages/zmq/eventloop/ioloop.py", line 162, in start
    super(ZMQIOLoop, self).start()
  File "/opt/anaconda3/lib/python3.5/site-packages/tornado/ioloop.py", line 887, in start
    handler_func(fd_obj, events)
  File "/opt/anaconda3/lib/python3.5/site-packages/tornado/stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "/opt/anaconda3/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
    self._handle_recv()
  File "/opt/anaconda3/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
    self._run_callback(callback, msg)
  File "/opt/anaconda3/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "/opt/anaconda3/lib/python3.5/site-packages/tornado/stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "/opt/anaconda3/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 276, in dispatcher
    return self.dispatch_shell(stream, msg)
  File "/opt/anaconda3/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 228, in dispatch_shell
    handler(stream, idents, msg)
  File "/opt/anaconda3/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 390, in execute_request
    user_expressions, allow_stdin)
  File "/opt/anaconda3/lib/python3.5/site-packages/ipykernel/ipkernel.py", line 196, in do_execute
    res = shell.run_cell(code, store_history=store_history, silent=silent)
  File "/opt/anaconda3/lib/python3.5/site-packages/ipykernel/zmqshell.py", line 501, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/opt/anaconda3/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2717, in run_cell
    interactivity=interactivity, compiler=compiler, result=result)
  File "/opt/anaconda3/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2821, in run_ast_nodes
    if self.run_code(code, result):
  File "/opt/anaconda3/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-9-f33e954aab9c>", line 6, in <module>
    Dense(1, init='normal', activation='sigmoid'),
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/models.py", line 268, in __init__
    self.add(layer)
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/models.py", line 327, in add
    output_tensor = layer(self.outputs[0])
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py", line 569, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py", line 632, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/engine/topology.py", line 164, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/layers/core.py", line 90, in call
    x = K.in_train_phase(K.dropout(x, self.p, noise_shape), x)
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 1877, in in_train_phase
    if learning_phase() is 1:
  File "/opt/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 73, in learning_phase
    name='keras_learning_phase')
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1587, in placeholder
    name=name)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2043, in _placeholder
    name=name)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
    op_def=op_def)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/opt/anaconda3/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1128, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'keras_learning_phase' with dtype bool
	 [[Node: keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

Issue Analytics

State:
Created 7 years ago
Reactions:5
Comments:5 (1 by maintainers)

Top GitHub Comments

4reactions

patyorkcommented, Jan 13, 2017

The error is self explanatory and well documented in both the Keras docs and a few other issues.

3reactions

zkk995commented, Jan 13, 2017

thanks @patyork.

done as follows.

def get_ith_layer_output(model, X, i, mode='test'):
    ''' see https://keras.io/getting-started/faq/#keras-faq-frequently-asked-keras-questions'''
    get_ith_layer = keras.backend.function(
        [model.layers[0].input, keras.backend.learning_phase()], [model.layers[i].output])
    layer_output = get_ith_layer([X, 0 if mode=='test' else 1])[0]
    return layer_output

Top Results From Across the Web

Keras, How to get the output of each layer? - Stack Overflow

You can easily get the outputs of any layer by using: model.layers[index].output. For all layers use this: from keras import backend as K ......

Where should I place dropout layers in a neural network?

Dropout was used after the activation function of each convolutional layer: CONV->RELU->DROP. So should they be placed after all layers, or only the...

Dropout: A Simple Way to Prevent Neural Networks from ...

We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and ...

Analysis of Dropout - P. Galeone's blog

The idea behind Dropout is to train an ensemble of DNNs and average the results of the whole ensemble instead of train a...

The dropout learning algorithm - ScienceDirect

When dropout is applied to the input units, we get:(15) E ( S i ) = ∑ j = 1 n ... layer...