Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ELU instead of ReLU in conv_dw_no_bn

See original GitHub issue

Hi, nice work on this repo!

I’m wondering that why do you use ELU instead of ReLU in con_dw_no_bw, while the conv_dw counterpart uses the regular ReLU: Is there a particular reason to use ELU? I didn’t see any mentioning of activation function in your paper or the original openpose paper.

Thank you!

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:1
  • Comments:10 (5 by maintainers)

github_iconTop GitHub Comments

Daniil-Osokincommented, May 23, 2019

Good question! For this particular network architecture there is no difference, it is from unfinished experiment. Nets with ReLU activations exposed “dead” neurons problem, so some percentage of neurons just never works. And the purpose of this work is to do lightweight net, yet maintain baseline accuracy. One of possible ways to reduce network complexity and save original capacity is to get rid of “dead” neurons with other activation function, so network can be more narrow (no “dead” neurons) but have same capacity (and accuracy). So here we’ve used ELU and the next thing is to reduce number of channels in these layers, but the time is left. I’ve taken this idea from RMNet paper: “Fast and Accurate Person Re-Identification with RMNet” by E. Izutov.

Daniil-Osokincommented, Mar 3, 2020

We do not. I’m think the best way is to mask out occluded points too, so network may predict them (if it smart enough), but we will not penalize it if it cannot do it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

ELU instead of ReLU in convdwnobn-程序媛/猿/员之家
程序媛/猿/员之家已搜集了关于[ELU instead of ReLU in convdwnobn]在实际运用中的解决方案,提供大量的ELU instead of ReLU in convdwnobn源码实例……
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found