Bidirectional RNNs?

I have an idea for adding bidirectional RNNs to Keras and I’m curious what the Keras devs think of it.

• Add a Reverse layer which simply slices its input tensor along the timestep dimension (e.g. X_input[:, ::-1]. This would preserve masking and slice masks in reverse as well.
• Add a Bidirectional class which takes an RNN class as a parameter and internally constructs the forward and backward instances, along with their merge. The backward instance can be Reverse(RNN(Reverse(x))).

How does that sound?

[codekansas]

Figured I’d add my 2 cents on this, using the functional API:

f_lstm = LSTM(n_lstm_dims)
b_lstm = LSTM(n_lstm_dims, go_backwards=True)
f_input = f_lstm(input)
b_input = b_lstm(input)
together = merge([f_input, b_input], mode='concat', concat_axis=1)

If return_sequences=True on the RNNs, change concat_axis=2 for time-wise concatenation.

However, due to the merge layer, this implementation doesn’t let you use Masking. Is there a way to do concatenation that supports masking? This seems like a common enough use case.

[carlthome]

Done, right?

https://keras.io/layers/wrappers/#bidirectional