Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

About the `Softmax` in the `CAM_Module`

See original GitHub issue

https://github.com/junfu1115/DANet/blob/799424e7fd9efa5723038ad0dc27c46682160e57/encoding/nn/attention.py#L75

Is your code trying to improve numerical stability? Maybe it should be in this form.

       energy_new = torch.max(energy, -1, keepdim=True)
       energy_new = energy_new[0].expand_as(energy)
       energy_new = energy - energy_new
       attention = self.softmax(energy_new)

Issue Analytics

State:
Created 4 years ago
Comments:5 (3 by maintainers)

Top GitHub Comments

1reaction

junfu1115commented, Apr 7, 2019

@lartpang Thanks, we have tried the form you provide, but the performance is not good

0reactions

junfu1115commented, Apr 8, 2019

@lartpang Thanks

Top Results From Across the Web

Softmax Function Definition | DeepAI

The softmax function is a function that turns a vector of K real values into a vector of K real values that sum...

Softmax function - Wikipedia

The softmax function, also known as softargmax or normalized exponential function, converts a vector of K real numbers into a probability distribution of...

A Simple Explanation of the Softmax Function - victorzhou.com

Softmax turns arbitrary real values into probabilities, which are often useful in Machine Learning. The math behind it is pretty simple: ...

Softmax Activation Function with Python

Softmax is a mathematical function that converts a vector of numbers into a vector of probabilities, where the probabilities of each value ...

Softmax Activation Function: Everything You Need to Know

The softmax activation function transforms the raw outputs of the neural network into a vector of probabilities, essentially a probability distribution over the ......