Permission denied: '/home/.astropy' when importing astropy in AWS EMR

Description

When I run a Jupyter notebook in AWS EMR using PySpark kernel I get a permission denied error when I run the first cell which contains an import of sbpy, it looks like astropy is trying to access ‘/home/.astropy’, which probably just doesn’t exist on the EC2 where the EMR notebook is running. How do we configure things so this won’t happen and astropy can initialize without trying to access this location?

Expected behavior

I expected for sbpy to be imported without an error.

Actual behavior

An error was encountered:
[Errno 13] Permission denied: '/home/.astropy'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sbpy/data/__init__.py", line 792, in <module>
    from .ephem import Ephem  # noqa: E402
  File "/usr/local/lib/python3.7/site-packages/sbpy/data/ephem.py", line 20, in <module>
    from astroquery.jplhorizons import Horizons
  File "/usr/local/lib/python3.7/site-packages/astroquery/jplhorizons/__init__.py", line 218, in <module>
    from .core import Horizons, HorizonsClass
  File "/usr/local/lib/python3.7/site-packages/astroquery/jplhorizons/core.py", line 1308, in <module>
    Horizons = HorizonsClass()
  File "/usr/local/lib/python3.7/site-packages/astroquery/jplhorizons/core.py", line 92, in __init__
    super(HorizonsClass, self).__init__()
  File "/usr/local/lib/python3.7/site-packages/astroquery/query.py", line 179, in __init__
    paths.get_cache_dir(), 'astroquery',
  File "/usr/local/lib64/python3.7/site-packages/astropy/config/paths.py", line 170, in get_cache_dir
    return os.path.abspath(_find_or_create_root_dir('cache', linkto, rootname))
  File "/usr/local/lib64/python3.7/site-packages/astropy/config/paths.py", line 297, in _find_or_create_root_dir
    os.mkdir(innerdir)
PermissionError: [Errno 13] Permission denied: '/home/.astropy'

Steps to Reproduce

1. Open EMR notebook in AWS AMR
2. In the first cell run import sbpy.data.Names

from sbpy.data import Names

System Details

Results of the above from EMR notebook cell: Linux-4.14.281-212.502.amzn2.x86_64-x86_64-with-glibc2.2.5 Python 3.7.10 (default, Jun 3 2021, 00:02:01) [GCC 7.3.1 20180712 (Red Hat 7.3.1-13)] Numpy 1.20.0 pyerfa 2.0.0.1 astropy 4.3.1 Scipy 1.7.3

[monocongo]

Yes agreed, it looks like this can be taken care of with some configuration for astropy on the EMR cluster. I don’t use astropy directly so this is my first exposure, I didn’t know there was this local .astropy directory being created, etc., I’m just using sbpy for converting asteroid designations, and I can do that outside of an EMR job so I’ve just done the work needing sbpy locally and it works fine. In any event I really appreciate the fast attention to this issue and for the guidance to the configuration options, @pllim. Thank you!

[monocongo]

Bmmk

On Sat, Jul 30, 2022, 3:42 PM P. L. Lim @.***> wrote:

Yes, indeed astropy creates a ~/.astropy for caching and config storing by default. I hope you could find a config that works for you. Good luck!

— Reply to this email directly, view it on GitHub https://github.com/astropy/astropy/issues/13512#issuecomment-1200298783, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKEIHQN737TT2KLTDMROKDVWWOUNANCNFSM547JR6JQ . You are receiving this because you modified the open/close state.Message ID: @.***>