Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

TPU auto_optim barrier is always True

See original GitHub issue

🐛 Bug description

In the auto_optim method when using TPUs should the barrier=True always be set to True when overriding the step. Unless I am understanding it incorrectly it should be changed depending on the number of TPU devices. Looking through the pytorch 1.5 docs it should only be set to True when using a single xla device.

EDIT: It looks like something changed from 1.5 to 1.7 with how xla works. pytorch 1.7 doesn’t use barrier but xm.mark_step() for single xla devices

Environment

PyTorch Version (e.g., 1.4):
Ignite Version (e.g., 0.3.0):
OS (e.g., Linux):
How you installed Ignite (conda, pip, source):
Python version:
Any other relevant information:

Issue Analytics

State:
Created 3 years ago
Comments:7 (3 by maintainers)

Top GitHub Comments

1reaction

vfdev-5commented, Jan 7, 2021

I will try to test the impact of the performance of barrier some time when I have the TPU resources as currently my 30 hours per week in Kaggle notebooks runs out pretty quickly.

Thanks ! Maybe, a good start can be on Google Colab and then on Kaggle 😃

0reactions

ryanwongsacommented, Jan 7, 2021

Sure I agree, I am happy to close the issue for now. I will try to test the impact of the performance of barrier some time when I have the TPU resources as currently my 30 hours per week in Kaggle notebooks runs out pretty quickly.

Top Results From Across the Web

Untitled

Mendelmax enclosure, Wilczy trop briggs, Cireson sccm, True node in eighth house, ... Monkey meme funny, Ice barrier yugioh cards, Cremolino ristorante, ...