Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BatchNorm in OP_SET version 7 has no mode attribute

See original GitHub issue

The BatchNormalization operator does not have mode attribute, so how do we figure out the difference between ‘training’ and ‘testing’ mode.

The OP_SET version 6 of the operator had a is_test attribute.

@bddppq @spidyDev

Issue Analytics

State:
Created 5 years ago
Comments:17 (17 by maintainers)

Top GitHub Comments

1reaction

tjingrantcommented, Jun 12, 2018

@ebarsoum it seems like the assumption that we are always in inference mode goes directly contradictory to a default momentum value of 0.9. In inference mode, momentum is effectively 1.0. How should we reconcile these contradictory default values?

0reactions

askhadecommented, Jun 9, 2021

Closing this issue since BatchNorm has been updated now to include the training mode : https://github.com/onnx/onnx/blob/master/docs/Operators.md#BatchNormalization

Top Results From Across the Web

8.5. Batch Normalization - Dive into Deep Learning

Like a dropout layer, batch normalization layers have different behaviors in training mode and prediction mode. Batch normalization is useful for regularization ...

The Effect of Batch Normalization on Deep Convolutional ...

Our experiments show that batch normalization indeed has positive effects on many aspects of neural networks but we cannot confirm significant convergence ...

tf.compat.v1.layers.batch_normalization | TensorFlow v2.11.0

When virtual_batch_size is not None , instead perform "Ghost Batch Normalization", which creates virtual sub-batches which are each normalized ...

How to Train Your ResNet 7: Batch Norm - Myrtle.ai

Things have not gone to plan (!) – there have been ample opportunities for optimisation, but batch norm has remained stubbornly in place....

API Reference :: NVIDIA Deep Learning cuDNN Documentation

The function does not support the provided configuration. CUDNN_STATUS_BAD_PARAM. At least one of the following conditions are met: The parameter mode has an ......