Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
h5py in a ppc64le compatible with torchmeta
See original GitHub issueI was trying to install torchmeta as I outlined here https://github.com/IBM/powerai/issues/269 and here (https://stackoverflow.com/questions/64049603/how-does-one-install-torchmeta-for-a-ppc64le-architecture-in-pytorch) but now I am getting I cannot install h5py. Is it possible to request support for it? Or point me to the version I need to be compatible with powerai 1.7.0 in a ppc64le architecture?
(my_new_env) [miranda9@hal-login ~]$ pip install torchmeta==1.3.1
Collecting torchmeta==1.3.1
Using cached torchmeta-1.3.1-py3-none-any.whl (144 kB)
Requirement already satisfied: requests in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (2.22.0)
Requirement already satisfied: torchvision<0.6.0,>=0.4.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (0.4.2)
Requirement already satisfied: torch<1.5.0,>=1.3.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (1.3.1)
Processing ./.cache/pip/wheels/87/f5/ad/9f04a48453875e8054c19f9fe3f50cbbe0c09b956835555019/Pillow-6.2.2-cp37-cp37m-linux_ppc64le.whl
Requirement already satisfied: numpy>=1.14.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (1.17.4)
Requirement already satisfied: tqdm>=4.0.0 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchmeta==1.3.1) (4.36.1)
Collecting h5py~=2.9.0
Using cached h5py-2.9.0.tar.gz (287 kB)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests->torchmeta==1.3.1) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests->torchmeta==1.3.1) (2020.6.20)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests->torchmeta==1.3.1) (1.25.10)
Requirement already satisfied: idna<2.9,>=2.5 in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from requests->torchmeta==1.3.1) (2.8)
Requirement already satisfied: six in ./.conda/envs/my_new_env/lib/python3.7/site-packages (from torchvision<0.6.0,>=0.4.0->torchmeta==1.3.1) (1.13.0)
Building wheels for collected packages: h5py
Building wheel for h5py (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/miranda9/.conda/envs/my_new_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ccg1oj0n
cwd: /tmp/pip-install-bpmeop26/h5py/
Complete output (1321 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-ppc64le-3.7
creating build/lib.linux-ppc64le-3.7/h5py
copying h5py/__init__.py -> build/lib.linux-ppc64le-3.7/h5py
copying h5py/h5py_warnings.py -> build/lib.linux-ppc64le-3.7/h5py
copying h5py/highlevel.py -> build/lib.linux-ppc64le-3.7/h5py
copying h5py/ipy_completer.py -> build/lib.linux-ppc64le-3.7/h5py
copying h5py/version.py -> build/lib.linux-ppc64le-3.7/h5py
creating build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/attrs.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/base.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/compat.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/dataset.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/datatype.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/dims.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/files.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/filters.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/group.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/selections.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/selections2.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
copying h5py/_hl/vds.py -> build/lib.linux-ppc64le-3.7/h5py/_hl
creating build/lib.linux-ppc64le-3.7/h5py/tests
copying h5py/tests/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/tests
copying h5py/tests/common.py -> build/lib.linux-ppc64le-3.7/h5py/tests
creating build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_attrs.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_attrs_data.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_base.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_dataset.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_datatype.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_dimension_scales.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_file.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_file_image.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_group.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_h5.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_h5d_direct_chunk_write.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_h5f.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_h5p.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_h5t.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_objects.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_selections.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
copying h5py/tests/old/test_slicing.py -> build/lib.linux-ppc64le-3.7/h5py/tests/old
creating build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/test_attribute_create.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/test_dataset_getitem.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/test_dataset_swmr.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/test_datatype.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/test_deprecation.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/test_dims_dimensionproxy.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/test_file.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/test_filters.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
copying h5py/tests/hl/test_threads.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl
creating build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
copying h5py/tests/hl/test_vds/__init__.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
copying h5py/tests/hl/test_vds/test_highlevel_vds.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
copying h5py/tests/hl/test_vds/test_lowlevel_vds.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
copying h5py/tests/hl/test_vds/test_virtual_source.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
running build_ext
Autodetected HDF5 1.10.2
********************************************************************************
Summary of the h5py configuration
Path to HDF5: None
HDF5 Version: '1.10.2'
MPI Enabled: False
Rebuild Required: True
********************************************************************************
Executing api_gen rebuild of defs
Executing cythonize()
[ 1/22] Cythonizing /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pyx
/tmp/pip-install-bpmeop26/h5py/.eggs/Cython-0.29.21-py3.7.egg/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pxd
tree = Parsing.p_module(s, pxd, full_module_name)
...
/home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it with " \
^
In file included from /tmp/pip-install-bpmeop26/h5py/h5py/defs.c:654:0:
/tmp/pip-install-bpmeop26/h5py/h5py/api_compat.h:27:18: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for h5py
Running setup.py clean for h5py
Failed to build h5py
DEPRECATION: Could not build wheels for h5py which do not use PEP 517. pip will fall back to legacy 'setup.py install' for these. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Installing collected packages: Pillow, h5py, torchmeta
Attempting uninstall: Pillow
Found existing installation: Pillow 7.1.2
Uninstalling Pillow-7.1.2:
Successfully uninstalled Pillow-7.1.2
Attempting uninstall: h5py
Found existing installation: h5py 2.8.0
Uninstalling h5py-2.8.0:
Successfully uninstalled h5py-2.8.0
Running setup.py install for h5py ... error
ERROR: Command errored out with exit status 1:
command: /home/miranda9/.conda/envs/my_new_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-hlwpfooj/install-record.txt --single-version-externally-managed --compile --install-headers /home/miranda9/.conda/envs/my_new_env/include/python3.7m/h5py
...
copying h5py/tests/hl/test_vds/test_lowlevel_vds.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
copying h5py/tests/hl/test_vds/test_virtual_source.py -> build/lib.linux-ppc64le-3.7/h5py/tests/hl/test_vds
running build_ext
Autodetected HDF5 1.10.2
********************************************************************************
Summary of the h5py configuration
Path to HDF5: None
HDF5 Version: '1.10.2'
MPI Enabled: False
Rebuild Required: True
********************************************************************************
Executing cythonize()
[ 1/22] Cythonizing /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pyx
/tmp/pip-install-bpmeop26/h5py/.eggs/Cython-0.29.21-py3.7.egg/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /tmp/pip-install-bpmeop26/h5py/h5py/_conv.pxd
...
warning: h5py/api_types_hdf5.pxd:730:6: 'H5Z_ERROR_EDC' redeclared
warning: h5py/api_types_hdf5.pxd:731:6: 'H5Z_DISABLE_EDC' redeclared
warning: h5py/api_types_hdf5.pxd:732:6: 'H5Z_ENABLE_EDC' redeclared
warning: h5py/api_types_hdf5.pxd:733:6: 'H5Z_NO_EDC' redeclared
building 'h5py.defs' extension
creating build/temp.linux-ppc64le-3.7
creating build/temp.linux-ppc64le-3.7/tmp
creating build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26
creating build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26/h5py
creating build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26/h5py/h5py
gcc -pthread -B /home/miranda9/.conda/envs/my_new_env/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DH5_USE_16_API -I./h5py -I/tmp/pip-install-bpmeop26/h5py/lzf -I/opt/local/include -I/usr/local/include -I/home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include -I/home/miranda9/.conda/envs/my_new_env/include/python3.7m -c /tmp/pip-install-bpmeop26/h5py/h5py/defs.c -o build/temp.linux-ppc64le-3.7/tmp/pip-install-bpmeop26/h5py/h5py/defs.o
In file included from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1830:0,
from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from /tmp/pip-install-bpmeop26/h5py/h5py/api_compat.h:26,
from /tmp/pip-install-bpmeop26/h5py/h5py/defs.c:654:
/home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it with " \
^
In file included from /tmp/pip-install-bpmeop26/h5py/h5py/defs.c:654:0:
/tmp/pip-install-bpmeop26/h5py/h5py/api_compat.h:27:18: fatal error: hdf5.h: No such file or directory
#include "hdf5.h"
^
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Rolling back uninstall of h5py
Moving to /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/h5py
from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/~5py
Moving to /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/h5py-2.8.0-py3.7.egg-info
from /home/miranda9/.conda/envs/my_new_env/lib/python3.7/site-packages/~5py-2.8.0-py3.7.egg-info
ERROR: Command errored out with exit status 1: /home/miranda9/.conda/envs/my_new_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bpmeop26/h5py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-hlwpfooj/install-record.txt --single-version-externally-managed --compile --install-headers /home/miranda9/.conda/envs/my_new_env/include/python3.7m/h5py Check the logs for full command output.
Issue Analytics
- State:
- Created 3 years ago
- Comments:9 (4 by maintainers)
Top Results From Across the Web
How does one install torchmeta for a ppc64le architecture in ...
I was trying to use torchmeta in a ppc64le architecture. ... it cannot find the right version of h5py compatible with what I...
Read more >torchmeta - PyPI
Torchmeta contains popular meta-learning benchmarks, fully compatible with both torchvision and PyTorch's DataLoader . Features. A unified interface for both ...
Read more >Package List — Spack 0.20.0.dev0 documentation
ants, pal, py-torchmeta ... hdf5-vol-external-passthrough, py-glymur, r-sass ... It includes a compatibility layer, FLAPACK, which includes complete LAPACK ...
Read more >PyTorch Dataloader for HDF5 data
I'm a newbie with HDF5, less so with PyTorch yet I found it hard to find guidelines regarding good practices to load data...
Read more >Torchmeta: A Meta-Learning library for PyTorch
We introduce Torchmeta, a library built on top of PyTorch that ... the development of models compatible with meta-learning algorithms.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I suggest creating a new env and rebulding it from scratch. It looks like you have a mix of conda default, conda from IBM, and pypi installed packages. Using a mix of packaging tools and sources it is easy to drive your env to a situation where it is orders of magnitude more work to understand what went wrong and fix it than it is to start fresh.
before I do that. Is there a way to force conda to reveal to me where it thinks everything is? (path to each package)