RuntimeError: cannot cache function '__shear_dense': no locator available for file '/usr/local/lib/python3.6/dist-packages/librosa/util/utils.py'

Describe the bug Cache issue on importing Librosa in a docker container when not run as root. ie.

I get the error when I run my docker container as - docker run -u $(id -u):$(id -g) ....

No error when I run - docker run ...

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-989066361697> in <module>
----> 1 import librosa

~usr/local/lib/python3.6/dist-packages/librosa/__init__.py in <module>
     10 # And all the librosa sub-modules
     11 from ._cache import cache
---> 12 from . import core
     13 from . import beat
     14 from . import decompose

~usr/local/lib/python3.6/dist-packages/librosa/core/__init__.py in <module>
    123 """
    124 
--> 125 from .time_frequency import *  # pylint: disable=wildcard-import
    126 from .audio import *  # pylint: disable=wildcard-import
    127 from .spectrum import *  # pylint: disable=wildcard-import

~usr/local/lib/python3.6/dist-packages/librosa/core/time_frequency.py in <module>
      9 import six
     10 
---> 11 from ..util.exceptions import ParameterError
     12 from ..util.deprecation import Deprecated
     13 

~usr/local/lib/python3.6/dist-packages/librosa/util/__init__.py in <module>
     75 """
     76 
---> 77 from .utils import *  # pylint: disable=wildcard-import
     78 from .files import *  # pylint: disable=wildcard-import
     79 from .matching import *  # pylint: disable=wildcard-import

~usr/local/lib/python3.6/dist-packages/librosa/util/utils.py in <module>
   1823 
   1824 @numba.jit(nopython=True, cache=True)
-> 1825 def __shear_dense(X, factor=+1, axis=-1):
   1826     '''Numba-accelerated shear for dense (ndarray) arrays'''
   1827 

~usr/local/lib/python3.6/dist-packages/numba/decorators.py in wrapper(func)
    191                           **dispatcher_args)
    192         if cache:
--> 193             disp.enable_caching()
    194         if sigs is not None:
    195             # Register the Dispatcher to the type inference mechanism,

~usr/local/lib/python3.6/dist-packages/numba/dispatcher.py in enable_caching(self)
    677 
    678     def enable_caching(self):
--> 679         self._cache = FunctionCache(self.py_func)
    680 
    681     def __get__(self, obj, objtype=None):

~usr/local/lib/python3.6/dist-packages/numba/caching.py in __init__(self, py_func)
    612     def __init__(self, py_func):
    613         self._name = repr(py_func)
--> 614         self._impl = self._impl_class(py_func)
    615         self._cache_path = self._impl.locator.get_cache_path()
    616         # This may be a bit strict but avoids us maintaining a magic number

~usr/local/lib/python3.6/dist-packages/numba/caching.py in __init__(self, py_func)
    347         else:
    348             raise RuntimeError("cannot cache function %r: no locator available "
--> 349                                "for file %r" % (qualname, source_path))
    350         self._locator = locator
    351         # Use filename base name as module name to avoid conflict between

RuntimeError: cannot cache function '__shear_dense': no locator available for file '/usr/local/lib/python3.6/dist-packages/librosa/util/utils.py'

To Reproduce sudo apt install libsndfile1 pip install librosa

import librosa

Expected behavior No error should occur.

Screenshots NA

Software versions*

Linux-5.3.0-53-generic-x86_64-with-Ubuntu-18.04-bionic
Python 3.6.9 (default, Apr 18 2020, 01:56:04)   [GCC 8.4.0]
NumPy 1.18.4
SciPy 1.4.1 
librosa 0.7.2 

INSTALLED VERSIONS
------------------
python: 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0]

librosa: 0.7.2

audioread: 2.1.8
numpy: 1.18.4
scipy: 1.4.1
sklearn: 0.23.1
joblib: 0.15.1
decorator: 4.4.2
six: 1.14.0
soundfile: 0.10.3
resampy: 0.2.2
numba: 0.49.1

numpydoc: None
sphinx: None
sphinx_rtd_theme: None
sphinxcontrib.versioning: None
sphinx-gallery: None
pytest: None
pytest-mpl: None
pytest-cov: None
matplotlib: 3.2.1
presets: None

Additional context I’m using tensorflow latest-gpu-jupyter (taken from official tf website) docker image to install these packages.

[seaniedan]

Thank you for this, hongwenxie. In case it helps anyone: I put the following in my Dockerfile: ENV NUMBA_CACHE_DIR=/tmp/

This worked, however I get this warning, which may be useful for debugging by someone clever: Matplotlib created a temporary config/cache directory at /tmp/matplotlib-de97eo76 because the default path (/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.

I then put THIS in my dockerfile, which sets up the directories with fully open permissions, and works:

environment variables for matplotlib

RUN mkdir -m 777 /tmp/NUMBA_CACHE_DIR /tmp/MPLCONFIGDIR ENV NUMBA_CACHE_DIR=/tmp/NUMBA_CACHE_DIR/ ENV MPLCONFIGDIR=/tmp/MPLCONFIGDIR/

[hongwenxie]

Add this to python file to solve this problem:

import os os.environ[ ‘NUMBA_CACHE_DIR’ ] = ‘/tmp/’