Questions of several implementation details

Hello @ku2482

May I ask you several implementation details and why you made these decisions?

1. In this line, you compute signs by comparing sa_quantiles[I] with sa_quantiles[i-1](except the first one). Why don’t you use values_1>0 as the signs?
2. In this line, you initialize the weights of the FractionProposalNetwork using Xavier initialize with gain=0.01. What makes you choose this initialization?

[toshikwa]

Hi @xlnwel

1 When F_Z^{-1}(w) is non-decreasing, the sign is +1 and gradients are the same as Proposition 1.

However, when F_Z^{-1}(w) is decreasing, the assumption of Proposition 1 doesn’t hold anymore. In this case, I correct the gradients setting the sign to -1.

2 It’s hand-designed by me.

The weights of the fraction proposal network are initialized so that initial probabilities are uniform as in QR-DQN

The author mentioned above in the paper, so I made the weight smaller so that initial probabilities are closer to the uniform distribution and the initialization was less influential.

Thanks 😃

[xlnwel]

Hi @ku2482

No problem. May I ask what your position is right now? Maybe we can share some thoughts after the challenge is completed.