TimeDistributed Wrapper not working with LSTM/GRU

Please make sure that the boxes below are checked before you submit your issue. If your issue is an implementation question, please ask your question on StackOverflow or join the Keras Slack channel and ask there instead of filing a GitHub issue.

Thank you!

• Check that you are up-to-date with the master branch of Keras. You can update with: pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

• If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with: pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

• Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

I am trying to apply word level attention to words in a document after passing the sentences through a GRU. However, the TimeDistributed Wrapper isn’t working with GRU/LSTM.

I get the following error

File "/home/##/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 569, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/home/##/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 632, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/home/##/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 164, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
File "/home/##/.local/lib/python2.7/site-packages/keras/layers/wrappers.py", line 129, in call
    y = self.layer.call(X)  # (nb_samples * timesteps, ...)
File "/home/##/.local/lib/python2.7/site-packages/keras/layers/recurrent.py", line 201, in call
    input_shape = K.int_shape(x)
File "/home/##/.local/lib/python2.7/site-packages/keras/backend/theano_backend.py", line 128, in int_shape
    raise Exception('Not a Keras tensor:', x)
Exception: ('Not a Keras tensor:', Reshape{3}.0)

The code snipped is written below

input_layer = Input(shape=(document_size, img_h,), dtype='int32', name='input_layer')
embedding_layer = TimeDistributed(Embedding(len(W2V), img_w, input_length=img_h, weights=[W2V], trainable=True, mask_zero=True))(input_layer)
gru_word = TimeDistributed(GRU(GRU_layers[0], return_sequences=True, activation=conv_non_linear))(embedding_layer)

[mahnerak]

@peschn I understand that, but K.rnn() calls step with non-keras tensors by design. In my opinion the problem should be fixed similary https://github.com/fchollet/keras/blob/master/keras/layers/wrappers.py#L186

Maybe it’s better to ask @fchollet before making pull-request.

[peschn]

@mahnerak the helper step is called within the method K.rnn(), but before it gets called, the dimshuffle (which is right at the beginning of the method: https://github.com/fchollet/keras/blob/0bc856f90a746ce3c8078f5ec4fb5156c88d8fdd/keras/backend/theano_backend.py#L1280) removes the _keras_shape, so when self.layer.call(x, **kwargs) within step is called, x doesn’t contain the shape information anymore, so call() will fail.