PyTorch Pruning with ProxylessNAS

Hi,

I am interested in using pruning with ProxylessNAS. So I added a new class like “MBInvertedConvLayer_Sparse” in ProxlyessNAS example that applies pruning in forward pass.

example:

 def forward(self, x):
        x = self.inverted_bottleneck(x)
        x = self.depth_conv(x)
        x = self.point_linear(x)
        return prune.ln_structured(nn.Conv2d, 'weight', 0.2)

However, I get the below error:

  File "~nni/retiarii/oneshot/pytorch/proxylessnas.py", line 74, in forward
    return ArchGradientFunction.apply(
  File "~nni/retiarii/oneshot/pytorch/proxylessnas.py", line 31, in forward
    return output.data
AttributeError: 'NoneType' object has no attribute 'data'

Any idea how I could hack proxylessNAS for it? Thanks!

[Lijiaoa]

@singagan Could you close this issue as completed?

[singagan]

@matluster Thanks for the quick reply. I see. Yes, I modified the input/output format to match my space. Thanks for pointing out the new examples. I am looking for a hardware-aware NAS. Maybe I can inject some hardware metrics in ENAS as well.