Distributed tests fail

Recently while working on ignite.contrib.metrics.regression tests, I was trying to add distributed tests to some metrics that didn’t have distributed tests. While i was running those tests when world_size > 1 the result computed by the metric implemented in ignite differs from the one computed by sklearn(if imported from sklearn) / numpy. And here are examples for that:

• Test wave hedges distance _test_distrib_integration() fails in case of world_size > 1 and if i didn’t comment all_gather() in _test_distrib_compute() it fails as well!

• Test fractional absolute error The same failures in the same two tests _test_distrib_compute() and _test_distrib_integration().

related: #1284

[vfdev-5]

@sdesrozis I think we can leave this task to @KickItLikeShika .

@KickItLikeShika check here: https://pytorch.org/ignite/metrics.html#metrics-and-distributed-computations and here : https://pytorch.org/ignite/distributed.html#ignite-distributed-utils for the list of collective ops. Please, try to figure out how it works by your own and ask if you tried everything and you become blocked for sometime. Thanks !

[sdesrozis]

Ah ah take a look, the metrics are not distributed. It means a reduction is needed in the implementation. I could fix it tomorrow.