Unexpected KeyError from lazyproperty on Python 3

Lately I’ve been writing / debugging a lot of code on Python 3.6 that uses astropy.utils.lazyproperty, and whenever there’s an error thrown in a lazyproperty function, I always see this KeyError from /astropy/utils/decorators.py", line 734, in __get__ before getting the traceback I expect:

>>> from astropy.utils import lazyproperty
>>> class Ham:
...     @lazyproperty
...     def ham(self):
...         1 / 0
... 
... 
>>> Ham().ham
>>> Traceback (most recent call last):
  File "/Users/deil/Library/Python/3.6/lib/python/site-packages/astropy-2.0.dev18379-py3.6-macosx-10.12-x86_64.egg/astropy/utils/decorators.py", line 734, in __get__
    return obj.__dict__[self._key]
KeyError: 'ham'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/deil/Library/Python/3.6/lib/python/site-packages/astropy-2.0.dev18379-py3.6-macosx-10.12-x86_64.egg/astropy/utils/decorators.py", line 736, in __get__
    val = self.fget(obj)
  File "<stdin>", line 4, in ham
ZeroDivisionError: division by zero

This KeyError doesn’t occur with a normal property:

>>> class Spam:
>>>     @property
...     def spam(self):
...         1 / 0
... 
... 
>>> Spam().spam
>>> Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in spam
ZeroDivisionError: division by zero

It also doesn’t occur (or at least isn’t visible to me as a user) on Python 2.7, there I get:

>>> from astropy.utils import lazyproperty
>>> class Spam:
...     @lazyproperty
...     def spam(self):
...         1 / 0
... 
>>> Spam().spam
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/deil/Library/Python/2.7/lib/python/site-packages/astropy-2.0.dev18379-py2.7-macosx-10.12-x86_64.egg/astropy/utils/decorators.py", line 736, in __get__
    val = self.fget(obj)
  File "<stdin>", line 4, in spam
ZeroDivisionError: integer division or modulo by zero

On Python 3.5 I see the same behaviour as on Python 3.6.

@embray - It looks like astropy.utils.lazyproperty is something you wrote a few years ago, and it hasn’t been touched since. Could you please comment if this is a bug or a feature?

[embray]

I wouldn’t be totally against changing this, as @cdeil suggested, to something like

_NotFound = object()

def __get__(self, obj, owner=None):
    try:
        val = obj.__dict__.get(self._key, _NotFound)
        if val is not _NotFound:
           return val
        else:
            ...
    except AttributeError:
        ...

The use of obj.__dict__.get(...) avoids a repeat dictionary lookup (as opposed to using if self._key in obj.__dict__:...). The custom sentinel singleton _NotFound is used since this is one of those rare cases where None could be a valid return value of the cached property.

I don’t think it’s worth changing myself, but I can see @cdeil’s point that it’s a bit distracting and untransparent. Astropy, being primarily a library, is a case where the line between developer and user is blurred, so users should “expect” and be able to sort of understand tracebacks. That said, lazyproperty is deep guts, and beyond easy comprehension for Python novices, so hiding its details, even in tracebacks, might be worthwhile if possible…

[cdeil]

Just to confirm that this is now fixed:

>>> import astropy
>>> astropy.version.githash
'd3bb55d9a57c7aa777f8e307f94899731ac23258'
>>> from astropy.utils import lazyproperty
>>> class Ham:
...     @lazyproperty
...     def ham(self):
...         1 / 0
... 
>>> Ham().ham
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/deil/Library/Python/3.6/lib/python/site-packages/astropy/utils/decorators.py", line 711, in __get__
    val = self.fget(obj)
  File "<stdin>", line 4, in ham
ZeroDivisionError: division by zero

@bsipocz and @embray - Thank you!