BatchNorm2d support
See original GitHub issueHi there,
I added a BatchNorm2d layer in the model and auto-LiRPA failed to parse the graph. Is it a bug?
$ python simple_training.py --num_epochs 3 --scheduler_opts "start=2,length=1"
Traceback (most recent call last):
File "simple_training.py", line 203, in <module>
main(args)
File "simple_training.py", line 170, in main
model = BoundedModule(model_ori, dummy_input, args.bound_opts, device=args.device)
File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/bound_general.py", line 22, in __init__
File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/bound_general.py", line 230, in _convert
File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/bound_general.py", line 119, in _convert_nodes
File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/parse_graph.py", line 135, in get_graph_params
File "/home/weilinxu/Envs/autoLiRPA/lib/python3.6/site-packages/auto_LiRPA-0.1-py3.6.egg/auto_LiRPA/parse_graph.py", line 65, in parse
AssertionError
Issue Analytics
- State:
- Created 3 years ago
- Comments:15 (8 by maintainers)
Top Results From Across the Web
BatchNorm2d — PyTorch 1.13 documentation
Applies Batch Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension) as described in the paper Batch Normalization: ......
Read more >BatchNorm2d: How to use the BatchNorm2d Module in PyTorch
BatchNorm2d - Use the PyTorch BatchNorm2d Module to accelerate Deep Network training by reducing internal covariate shift.
Read more >How to use the torch.nn.BatchNorm2d function in torch - Snyk
To help you get started, we've selected a few torch.nn.BatchNorm2d examples, based on popular ways it is used in public projects.
Read more >How to implement Batchnorm2d in Pytorch myself?
I'm trying to implement Batchnorm2d() layer with: class BatchNorm2d(nn. ... I don't know how to fix it, please help. Thanks!!
Read more >Different speed between BatchNorm1d and BatchNorm2d
Hi I assume BatchNorm1d and BatchNorm2d (or 3d) should have same performance. However when normalizing a tensor of shape (L, C) = (B*H*W, ......
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@mzweilin @Harry24k @Kozon2015 We release a new version of
auto_LiRPA
today, which comes with batchnorm support. An example ofresnet18
with batchnorm can be found here: https://github.com/KaidiXu/CROWN-GENERAL/blob/master/examples/vision/models/resnet18.py You can usecifar_training.py
to train this model:It takes about 80 seconds to train 1 epoch on 4x 1080 Ti GPUs using CROWN-IBP plus loss fusion. The default parameters are probably not optimal for this model (we did not tune it), so you probably need some tuning to obtain good results on this model.
@Harry24k Thank you for the detailed report! We will look into this. We plan to release a new version of
auto_LiRPA
soon which will try to fix this problem.