Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
softmax_mask
See original GitHub issueThis repo is written very nicely, thanks for sharing!
My question is not related with this system, just more about understanding RNet. I don’t get why you use softmax_mask() before softmax operations. Could you explain more? Thanks again!
Issue Analytics
- State:
- Created 5 years ago
- Comments:6
Top Results From Across the Web
Apply mask softmax - PyTorch Forums
Hi everyone, I try to implement the following function: [image] At this stage, I have e.g. a tensor [[1,0,3], [0, 1, 2], [3,...
Read more >Masked Softmax in PyTorch - gists · GitHub
It is to ensure that the sum used for normalization excludes non-masked elements. This implementation is not ideal for the latest pytorch (v1.6)....
Read more >Tensorflow softmax does not ignore masking value
The only way to get a zero output from a softmax() is to pass a very small float value. If you set the...
Read more >Softmax layer - Keras
Softmax activation function. Example without mask ... inputs: The inputs, or logits to the softmax layer. mask: A boolean mask of the same...
Read more >tf.keras.layers.Softmax | TensorFlow v2.11.0
A boolean mask of the same shape as inputs . Defaults to None . The mask specifies 1 to keep and 0 to...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Assume the length is 2, the output is [1, 1, 0], then after softmax is [0.42, 0.42, 0.16], wrong! Masking it to [1, 1, -1e30] before softmax, then result is [1/2, 1/2, 0] That’s softmax_mask used for.
Thanks this makes sense.