Location sensitive attention shouldn't use preprocessed memory?

Keith, I see that you’re also using the keys (preprocessed memory) to compute the location sensitive attention scores. Shouldn’t it be just the processed query and the processed location?

https://github.com/keithito/tacotron/blob/c94ab2757d52e4294dcd6a8da03f49d251b2dec4/models/attention.py#L61

[keithito]

Yes – sorry for the confusion. I’ll update the naming and comments to be more clear about this.

[pravn]

This is really nice code. Thanks very much.

So, if I can clarify this (advance apologies if it is stating the obvious …) fi = F ∗ αi−1. (8): This corresponds to f = self.location_conv(expanded_alignments) # [N, T_in, 10]

With ‘k’ vectors being the number of filters, and ‘r’ being the filter size.

The second operation corresponding to equation (9): ei,j = vT tanh(W si−1 + V hj + Ufi,j + b) (9)

Does that correspond to Uf in the code below? processed_location = self.location_layer(f) # [N, T_in, num_units]

With equation (9) being computed as return tf.reduce_sum(v * tf.tanh(keys + processed_query + processed_location), [2])

The paper says “First, we extract k vectors fi,j ∈ R k for every position j of the previous alignment αi−1 by convolving it with a matrix F ∈ R k×r”.

I am assuming that we arbitrarily choose the size of the vectors == hidden dim, as the paper does not specify what this size should be, only saying that we need ‘k’ such vectors. :