alpha in Log space.

Hi. I’m just wondering why alpha get updated in ‘log’ space rather than in its own value. Was doing that stable for entire training? Or any other reason?

# In https://github.com/medipixel/rl_algorithms/blob/master/algorithms/sac/agent.py

 # train alpha
        if self.hyper_params["AUTO_ENTROPY_TUNING"]:
            alpha_loss = (
                -self.log_alpha * (log_prob + self.target_entropy).detach()
            ).mean()

            self.alpha_optimizer.zero_grad()
            alpha_loss.backward()
            self.alpha_optimizer.step()

            alpha = self.log_alpha.exp()
        else:
            alpha_loss = torch.zeros(1)
            alpha = self.hyper_params["W_ENTROPY"]

[Curt-Park]

That’s because alpha should be greater than or equal to zero due to the dual problem’s constraint. see (13).

[Junyoungpark]

Sure. Thanks again.