Cannot use VideoReader due to ffmpeg issue

Hey experts,

Recently I’ve been trying to use the torchvision.io.VideoReader API on my MacOS machine.

Description

At first, I was trying to import the API directly, but hitting this issue:

  File "/Users/sallysyw/workspace/vision/gallery/plot_video_api.py", line 36, in <module>
    video = torchvision.io.VideoReader(video_path, stream)
  File "/Users/sallysyw/workspace/vision/torchvision/io/__init__.py", line 104, in __init__
    raise RuntimeError(
RuntimeError: Not compiled with video_reader support, to enable video_reader support, please install ffmpeg (version 4.2 is currently supported) andbuild torchvision from source.

Then I realized that I need to manually install ffmpeg package to get this work. Firstly I tried conda install:

conda install ffmpeg=4.2=h677a3f5_0

After this is done, I re-built the torchvision package with

MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py develop

Then I tried

import torchvision

this time I’m hitting a new error:

libc++abi: terminating with uncaught exception of type c10::Error: Type c10::tagged_capsule<vision::video::Video> could not be converted to any of the known types.
Exception raised from operator() at /Users/sallysyw/opt/anaconda3/lib/python3.8/site-packages/torch/include/ATen/core/jit_type.h:1606 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >) + 98 (0x102646e12 in libc10.dylib)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 106 (0x10264552a in libc10.dylib)
frame #2: c10::detail::getTypePtr_<c10::tagged_capsule<vision::video::Video> >::call()::'lambda'()::operator()() const + 263 (0x129f49497 in video_reader.so)
frame #3: c10::detail::getTypePtr_<c10::tagged_capsule<vision::video::Video> >::call() + 27 (0x129f492eb in video_reader.so)
frame #4: c10::detail::infer_schema::(anonymous namespace)::createArgumentVector(c10::ArrayRef<c10::detail::infer_schema::ArgumentDef>) + 216 (0x10e27c508 in libtorch_cpu.dylib)
frame #5: c10::detail::infer_schema::make_function_schema(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, c10::ArrayRef<c10::detail::infer_schema::ArgumentDef>, c10::ArrayRef<c10::detail::infer_schema::ArgumentDef>) + 123 (0x10e27c2ab in libtorch_cpu.dylib)
frame #6: torch::jit::Function* torch::class_<vision::video::Video>::defineMethod<torch::class_<vision::video::Video>& torch::class_<vision::video::Video>::def<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(torch::detail::types<void, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::initializer_list<torch::arg>)::'lambda'(c10::tagged_capsule<vision::video::Video>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >)>(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::initializer_list<torch::arg>) + 477 (0x129f4872d in video_reader.so)
frame #7: torch::class_<vision::video::Video>& torch::class_<vision::video::Video>::def<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(torch::detail::types<void, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::initializer_list<torch::arg>) + 114 (0x129f35242 in video_reader.so)
frame #8: _GLOBAL__sub_I_video.cpp + 226 (0x129f5b202 in video_reader.so)
frame #9: 0x0 + 4338874055 (0x1029df6c7 in ???)
frame #10: 0x0 + 4338875090 (0x1029dfad2 in ???)
frame #11: 0x0 + 4338853046 (0x1029da4b6 in ???)
frame #12: 0x0 + 4338844271 (0x1029d826f in ???)
frame #13: 0x0 + 4338844432 (0x1029d8310 in ???)
frame #14: 0x0 + 4338780690 (0x1029c8a12 in ???)
frame #15: 0x0 + 4338825966 (0x1029d3aee in ???)
frame #16: dlopen_internal(char const*, int, void*) + 185 (0x7fff203edcb4 in libdyld.dylib)
frame #17: dlopen + 28 (0x7fff203dc09e in libdyld.dylib)
frame #18: py_dl_open + 135 (0x102303ca7 in _ctypes.cpython-38-darwin.so)
<omitting python frames>

[1]    216 abort      python

I also tried the following options:

1. Download ffmpeg-4.2.4.tar.xz on their website and build it manually
2. Uninstall the conda package and use brew to install ffmpeg again

However, neither of them solves the issue, I’m still getting the python frames error. Can anyone help? Thanks a lot!

Environment

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

Results:

Collecting environment information...
PyTorch version: 1.10.0.dev20210728
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 11.4 (x86_64)
GCC version: Could not collect
Clang version: 11.0.0 (clang-1100.0.33.17)
CMake version: Could not collect
Libc version: N/A

Python version: 3.8.8 (default, Apr 13 2021, 12:59:45)  [Clang 10.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.1
[pip3] numpydoc==1.1.0
[pip3] pytorch-sphinx-theme==0.0.24
[pip3] torch==1.10.0.dev20210728
[pip3] torchvision==0.11.0a0+b29ed34
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.2.0           hecd8cb5_269
[conda] mkl-service               2.3.0            py38h9ed2024_1
[conda] mkl_fft                   1.3.0            py38h4a7008c_2
[conda] mkl_random                1.2.1            py38hb2f4e1b_2
[conda] numpy                     1.20.1           py38hd6e1bb9_0
[conda] numpy-base                1.20.1           py38h585ceec_0
[conda] numpydoc                  1.1.0              pyhd3eb1b0_1
[conda] pytorch                   1.10.0.dev20210728         py3.8_0    pytorch-nightly
[conda] pytorch-sphinx-theme      0.0.24                    dev_0    <develop>
[conda] torch                     1.9.0                    pypi_0    pypi
[conda] torchvision               0.11.0a0+b29ed34           dev_0    <develop>

[vmoens]

One way to go around it is to install ffmpeg 4.4 (eg with brew or from source) and comment out the ffmpeg check part in the setup.py (l351-362). With that I can read videos flawlessly

[bjuncek]

@NicolasHug for FFMPEG, 4.3 is known to have a segfault, that’s why we’re only supporting 4.2 for now. I’ll try to get my hands on a mac to investigate @sallysyw 's issue today