Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Does MDEQ have different inference results for different batch sizes?

See original GitHub issue

I’m running some experiments with MDEQ on ImageNet validation set and I get different activations (variable new_z1 in the mdeq_core.py) for the DEQ layer for different batch sizes. I can see in the broyden function that there’s no loop over the batch but since I’m not familiar with Broyden’s method and its implementation, I do not know if different images (within a batch) can interfere with each other or not directly or indirectly (by having an effect on the number of iterations in the solver for instance). Should I run inference on 1 image at a time?

Issue Analytics

State:
Created 2 years ago
Comments:6 (3 by maintainers)

Top GitHub Comments

1reaction

jerrybai1995commented, Feb 17, 2022

Can you check if this issue solves the problem?

0reactions

undercutspikycommented, Feb 18, 2022

Yes, that solved the issue. Thanks a lot!

Top Results From Across the Web

The batch size can affect inference results | OpenReview

It experimentally finds that the different batch sizes during training and inference will affect the model performance due to the matrix ...

Different batch size different result with inception_v2 #11295

During testing, if the batchsize=128, everything is ok. However, if the batchsize is smaller than 128 the results are different.

Multiscale Deep Equilibrium Models - Vladlen Koltun

We propose a new class of implicit networks, the multiscale deep equilibrium model. (MDEQ), suited to large-scale and highly hierarchical pattern ...

Sampling Strategies and Statistics Training Materials

Appropriate sampling strategies differ based on the sampling objectives (e.g, FACILITY characterization, verification of remediation, comparison to criteria ...

Training Graph Neural Networks with 1000 Layers - arXiv

We examine different iteration thresholds for DEQ-GNN with a channel size of 64 (DEQ-GNN-64) and a channel size of 224 (DEQ-. GNN-224) in...