Passing a bound function fails with 0.7.0+

We have started having a problem with 0.7.0+ due to changes in [1] where the input function is no longer wrapped.

The shortest replication of what’s going on I can make is the following

from dogpile.cache import make_region

region = make_region().configure('dogpile.cache.memory', expiration_time = 3600)

def _cache_on_arguments(*cache_on_args, **cache_on_kwargs):
    def _inner_cache_on_arguments(func):
        def _cache_decorator(obj, *args, **kwargs):
            dogpile_decorator = region.cache_on_arguments(
                *cache_on_args, **cache_on_kwargs)
            self_method = func.__get__(obj, type(obj))
            wrapped_func = dogpile_decorator(self_method)
            return wrapped_func(*args, **kwargs)
        return _cache_decorator
    return _inner_cache_on_arguments

class MainClass(object):
    @_cache_on_arguments()
    def function(self):
        print("A function")

cls = MainClass()
cls.function()

When you run this under 0.7.0 onwards you get

$ ./env/bin/python3 ./test-dp.py 
Traceback (most recent call last):
  File "./test-dp.py", line 30, in <module>
    cls.function()
  File "./test-dp.py", line 15, in _cache_decorator
    wrapped_func = dogpile_decorator(self_method)
  File "/home/iwienand/tmp/dogpile/env/lib64/python3.7/site-packages/dogpile/cache/region.py", line 1304, in cache_decorator
    user_func.set = set_
AttributeError: 'method' object has no attribute 'set'

When you run it on 0.6.8 you see “A function” printed.

The old cache_on_arguments would take the input callable, and then wrap it in it’s decorate function which then actually called out to get_or_create(key, creator, ...). It would then set the various helpers set, invalidate, refresh, get, original on the internal decorate function.

This changed with [1] where set,invalidate,get,refresh etc are being directly set on user_func. This is a problem when user_func is actually a bound method (what we found from func.__get__(obj, type(obj))) and not a function … we can’t set arbitrary things on it.

I’m a bit stumped on all this at the moment. Our use is certainly “interesting” (and yes, I’m reminded of [2] 😃. We are tracking this in [3]. The point of [1] appears to be to get rid of the wrapping to avoid signature changes, so I’m not sure about adding it back in, but I’m also not sure how to fix what we have 😕

Thoughts on this are very welcome

[1] https://gerrit.sqlalchemy.org/#/c/sqlalchemy/dogpile.cache/+/996/4/dogpile/cache/region.py [2] https://xkcd.com/1172/ [3] https://storyboard.openstack.org/#!/story/2004605

[ankitpatel96]

I kind of agree with @morganfainberg here - openstack’s use case is pretty strange and the workaround linked doesn’t look too arduous.
On top of that, decorator, doesn’t support bound methods even if I reorder the set/refresh/etc.

[morganfainberg]

I think dogpile is 100% sane here. Realistically dogpile shouldn’t need to assume that the passed in object isn’t a bound instanct. What OpenStackSDK is doing is very strange. You can see a quick fix within SDK here: https://review.openstack.org/#/c/625370/

I don’t feel strongly one way or another on revert/use decorate/etc. We could also conditionally add a subsequent simple wrap if needed if the object is a bound method.

While this is a change in behavior, I don’t think it’s an unreasonable change in behavior from pre 0.7.0. It’s weird to pass bound methods into a decorator in most cases.