ImportError:Extension horovod.torch has not been built

from horovod.tensorflow import allreduce_async_, synchronize

The program runs at the line above break off. The error info as below：

Traceback (most recent call last):
  File "/GPUFS/nudt_chkwu_2/kfhu/horovod-0.19.2/horovod/torch/__init__.py", line 32, in <module>
    __file__, 'mpi_lib_v2')
  File "/GPUFS/nudt_chkwu_2/kfhu/horovod-0.19.2/horovod/common/util.py", line 56, in check_extension
    'Horovod with %s=1 to debug the build error.' % (ext_name, ext_env_var))
ImportError: Extension horovod.torch has not been built.  If this is not expected, reinstall Horovod with HOROVOD_WITH_PYTORCH=1 to debug the build error.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/GPUFS/nudt_chkwu_2/kfhu/horovod-0.19.2/horovod/torch/__init__.py", line 35, in <module>
    __file__, 'mpi_lib', '_mpi_lib')
  File "/GPUFS/nudt_chkwu_2/kfhu/horovod-0.19.2/horovod/common/util.py", line 56, in check_extension
    'Horovod with %s=1 to debug the build error.' % (ext_name, ext_env_var))
ImportError: Extension horovod.torch has not been built.  If this is not expected, reinstall Horovod with HOROVOD_WITH_PYTORCH=1 to debug the build error.

Can you give me a resolution? Appreciate for your help!

[KevvinHoo]

There is no problem when I used compression = hvf.Compression.fp16 if args.fp16 allreduce else hvd.Compression.none, which used in the example program supported by Horovod, instead of grc = Allgather(TopKCompressor(0.3), ResidualMemory(), hvd.size()). That bug you mentioned above already be fixed before running the training script. It is the truth that I haven’t apply the patch, as I don’t know how to make it. Could you tell me the details about this patch?

Best regards

[hangxu0304]

No need. Just modify the related pytorch files.

在 2020年5月28日，16:12，Tonyhukaifan [e-mail] 写道：

Never mind - I have read the file named horovd 0.18.2-patch. Maybe I know how to apply this patch. The - indicate to delete and the + is to add, right? And is it needed to make a change to the module file about the TensorFlow when I used the PyTorch framework?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

–

This message and its contents, including attachments are intended solely for the original recipient. If you are not the intended recipient or have received this message in error, please notify me immediately and delete this message from your computer system. Any unauthorized use or distribution is prohibited. Please consider the environment before printing this email.