Activation selection within the bottlenecks in the network

if relu:
    activation = nn.ReLU()
else:
    activation = nn.PReLU()

Does doing this ensure that PReLU weights are unique for each instance of activation within the bottlenecks? While trying to trace this network with torch.jit, it gives errors regarding shared weights by nn.PReLU layers within the submodules. Perhaps this should be implemented with copy.deepcopy for all instances?

To follow the original paper more closely, the number of channels can be specified for each PReLU instance to learn a weight per channel as shown here.

[heethesh]

@davidtvs Thanks, the tracing works fine now!

[davidtvs]

For future reference - the fix is now on the master branch