Is there any reason for `TimeDistributedDense`?

Similar to how I removed time_distributed_softmax, and just made it always apply softmax to the last axis (i.e. the nb_dimensions) axis, is there any reason that I don’t make Dense have the same behavour, so you can pass it either a (nb_samples, nb_dims) or a (nb_samples, nb_timesteps, nb_dims) matrix? Seems like it reduces complexity with no downside.

[falaktheoptimist]

So, from this discussion I glean that: model.add(LSTM(512, input_shape=(maxlen, len(chars)),return_sequences=True)) model.add(LSTM(512, return_sequences=True)) #- original model.add(Dropout(0.2)) model.add(Dense(len(chars))) model.add(Activation('softmax'))

and

model.add(LSTM(512, input_shape=(maxlen, len(chars)),return_sequences=True)) model.add(LSTM(512, return_sequences=True)) #- original model.add(Dropout(0.2)) model.add(TimeDistributed(Dense(len(chars)))) model.add(Activation('softmax'))

represent the exact same model. I also checked the summary and the number of parameters are also exactly the same which leads me to conclude that I can probably do away with TimeDistribute block altogether without affecting the model in any way. Please correct me here if this is incorrect

[DomHudson]

@falaktheoptimist Did you ever work this out? I’m wondering the same thing - the output dimensions, loss and accuracy are identical for me if I replace my final TimeDistributed(Dense()) layer with just a Dense layer.

EDIT: I’m wondering if https://github.com/fchollet/keras/pull/7554 is related.