ELU instead of ReLU in conv_dw_no_bnSee original GitHub issue
Hi, nice work on this repo!
I’m wondering that why do you use
ELU instead of
con_dw_no_bw, while the
conv_dw counterpart uses the regular
Is there a particular reason to use
ELU? I didn’t see any mentioning of activation function in your paper or the original openpose paper.
- Created 4 years ago
- Comments:10 (5 by maintainers)
Top GitHub Comments
Good question! For this particular network architecture there is no difference, it is from unfinished experiment. Nets with
ReLU activations exposed “dead” neurons problem, so some percentage of neurons just never works. And the purpose of this work is to do lightweight net, yet maintain baseline accuracy. One of possible ways to reduce network complexity and save original capacity is to get rid of “dead” neurons with other activation function, so network can be more narrow (no “dead” neurons) but have same capacity (and accuracy). So here we’ve used
ELU and the next thing is to reduce number of channels in these layers, but the time is left. I’ve taken this idea from RMNet paper: “Fast and Accurate Person Re-Identification with RMNet” by E. Izutov.
We do not. I’m think the best way is to mask out occluded points too, so network may predict them (if it smart enough), but we will not penalize it if it cannot do it.