Torchaudio Compilation Error

🐛 Bug

When trying to compile torchaudio from source after python setup.py install within anaconda, it fails with an error that configure: error: cannot find flac (see below for full information).

To Reproduce

Steps to reproduce the behavior:

1. Compile PyTorch from source in conda
2. Compile Torchaudio from source in conda

See code as below:

module load cuda/10.1
conda create -n audio-master python=3.7
conda activate audio-master
# install cuda with needed version
conda install -c pytorch magma-cuda101

conda install numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing
git clone --recursive https://github.com/pytorch/pytorch
cd pytorch
git submodule sync
git submodule update --init --recursive
git submodule sync
git submodule update --init --recursive

python setup.py clean --all
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py install

cd ..
python -c "import torch; print(torch.__version__); print(torch.cuda.is_available());"

conda install -c conda-forge sox
git clone https://github.com/pytorch/audio
cd audio

python setup.py clean --all              
python setup.py install

cd ..
python -c "import torchaudio;"

Error message:

After python setup.py install for torchaudio, it shows:

Building sox
configure: error: in `/private/home/jimchen90/audio/third_party/tmp/sox-14.4.2':
configure: error: cannot find flac
See `config.log' for more details
subprocess.CalledProcessError: Command '['/private/home/jimchen90/audio/build_tools/setup_helpers/build_third_party.sh']' returned non-zero exit status 1.

Expected behavior

Torchaudio can be imported outside the source folder.

Environment

• PyTorch Version: 1.6.0a0+78acc9d
• OS: Linux
• How you installed PyTorch: Conda
• Build command you used : As described above
• Python version: 3.7
• CUDA/cuDNN version: CUDA 10.1

Additional context

With the help of Moto @mthrok , torchaudio can be imported after adding an external sox file in the audio directory by:

conda uninstall sox
rm -rf third_party
touch .use_external_sox
python setup.py clean develop

The result of python -m torch.utils.collect_env

Collecting environment information...
PyTorch version: 1.6.0a0+78acc9d
Is debug build: No
CUDA used to build PyTorch: 10.1

OS: Ubuntu 18.04.3 LTS
GCC version: (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
CMake version: version 3.14.0

Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 10.1.105
GPU models and configuration:
GPU 0: Quadro GP100
GPU 1: Quadro GP100

Nvidia driver version: 418.116.00
cuDNN version: Could not collect

Versions of relevant libraries:
[pip] numpy==1.18.1
[pip] torch==1.6.0a0+78acc9d
[pip] torchaudio==0.6.0a0+fd73d8f
[conda] blas                      1.0                         mkl
[conda] magma-cuda101             2.5.2                         1    pytorch
[conda] mkl                       2020.1                      217
[conda] mkl-include               2020.1                      217
[conda] mkl-service               2.3.0            py37he904b0f_0
[conda] mkl_fft                   1.0.15           py37ha843d7b_0
[conda] mkl_random                1.1.1            py37h0573a6f_0
[conda] numpy                     1.18.1           py37h4f9e942_0
[conda] numpy-base                1.18.1           py37hde5b4d6_1
[conda] torch                     1.6.0a0+78acc9d          pypi_0    pypi
[conda] torchaudio                0.6.0a0+fd73d8f           dev_0    <develop>

Related issues can be seen in https://github.com/pytorch/audio/issues/666 and https://github.com/pytorch/audio/pull/669. Thank @vincentqb and @mthrok for helping me solve this issue.

[mthrok]

fixed by #750 and #755

[mthrok]

The above solution led to another issue with

test/test_sox_io.py:7: in <module>
    import torchaudio
torchaudio/__init__.py:1: in <module>
    from .extension import (
torchaudio/extension/__init__.py:7: in <module>
    _init_extension()
torchaudio/extension/extension.py:11: in _init_extension
    _init_script_module(ext)
torchaudio/extension/extension.py:18: in _init_script_module
    torch.classes.load_library(path)
../pytorch/torch/_classes.py:46: in load_library
    torch.ops.load_library(path)
../pytorch/torch/_ops.py:105: in load_library
    ctypes.CDLL(path)
/home/moto/conda/envs/PY3.8-cuda101/lib/python3.8/ctypes/__init__.py:373: in __init__
    self._handle = _dlopen(self._name, mode)
E   OSError: /scratch/moto/torchaudio/torchaudio/_torchaudio.so: undefined symbol: op_pcm_seek

This was caused by the fact opusfile found in /usr/include/opus / /usr/lib/libopusfile.

configure:14728: checking for OPUS
configure:14735: $PKG_CONFIG --exists --print-errors "opusfile"
configure:14738: $? = 0
configure:14752: $PKG_CONFIG --exists --print-errors "opusfile"
configure:14755: $? = 0
...
pkg_cv_OPUS_CFLAGS=-I/usr/include/opus
pkg_cv_OPUS_LIBS=-lopusfile

Similarly, adding --with-opus=no to build_sox function resolved this.