question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BatchNorm2d support

See original GitHub issue

Hi there,

I added a BatchNorm2d layer in the model and auto-LiRPA failed to parse the graph. Is it a bug?

$ python simple_training.py --num_epochs 3 --scheduler_opts "start=2,length=1"
Traceback (most recent call last):
  File "simple_training.py", line 203, in <module>
    main(args)
  File "simple_training.py", line 170, in main
    model = BoundedModule(model_ori, dummy_input, args.bound_opts, device=args.device)
  File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/bound_general.py", line 22, in __init__
  File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/bound_general.py", line 230, in _convert
  File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/bound_general.py", line 119, in _convert_nodes
  File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/parse_graph.py", line 135, in get_graph_params
  File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/parse_graph.py", line 65, in parse
AssertionError

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:15 (8 by maintainers)

github_iconTop GitHub Comments

2reactions
huanzhang12commented, Jul 3, 2020

@mzweilin @Harry24k @Kozon2015 We release a new version of auto_LiRPA today, which comes with batchnorm support. An example of resnet18 with batchnorm can be found here: https://github.com/KaidiXu/CROWN-GENERAL/blob/master/examples/vision/models/resnet18.py You can use cifar_training.py to train this model:

python cifar_training.py --model resnet18

It takes about 80 seconds to train 1 epoch on 4x 1080 Ti GPUs using CROWN-IBP plus loss fusion. The default parameters are probably not optimal for this model (we did not tune it), so you probably need some tuning to obtain good results on this model.

1reaction
huanzhang12commented, Jun 23, 2020

@Harry24k Thank you for the detailed report! We will look into this. We plan to release a new version of auto_LiRPA soon which will try to fix this problem.

Read more comments on GitHub >

github_iconTop Results From Across the Web

BatchNorm2d — PyTorch 1.13 documentation
Applies Batch Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) as described in the paper Batch Normalization: ......
Read more >
BatchNorm2d: How to use the BatchNorm2d Module in PyTorch
BatchNorm2d - Use the PyTorch BatchNorm2d Module to accelerate Deep Network training by reducing internal covariate shift.
Read more >
How to use the torch.nn.BatchNorm2d function in torch - Snyk
To help you get started, we've selected a few torch.nn.BatchNorm2d examples, based on popular ways it is used in public projects.
Read more >
How to implement Batchnorm2d in Pytorch myself?
I'm trying to implement Batchnorm2d() layer with: class BatchNorm2d(nn. ... I don't know how to fix it, please help. Thanks!!
Read more >
Different speed between BatchNorm1d and BatchNorm2d
Hi I assume BatchNorm1d and BatchNorm2d (or 3d) should have same performance. However when normalizing a tensor of shape (L, C) = (B*H*W, ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found