Cython functions not bound as methods

In Python, one can do

class A:
    pass

def foo(self):
    print self

A.foo = foo
print A().foo()

and foo will get bound as expected. However, Cython creates PyCFunctions which do not properly bind when accessed (possibly by design). This is the root issue behind http://trac.cython.org/ticket/478, and will become more evident once closures start getting used.

PyCFunction does not allow subclassing. The only reasonable fix I see is to create a new type that wraps PyCFunction but behaves like PyFunction. This will have indirection overhead. To avoid overhead, one alternative is for our new type to have the same struct and slots as PyCFunction (with the exception of tp_name, tp_descr_get, tp_alloc, and tp_dealloc of course). This still may have overhead as the Python compiler optimizes for PyCFunction calls–perhaps the user should be allowed to control this (as part of an “ultra pure, I don’t care what cost” mode…)?

Migrated from http://trac.cython.org/ticket/494

[robertwb]

bfroehle changed cc to bfroehle commented

I’ve run into this before and have also asked about it on StackOverflow. In general it’s pretty easy to work around in Python 2:

A.foo = types.MethodType(foo, None, A)

But somewhat troublesome in Python 3. The best suggestion I got was:

A.foo = functools.update_wrapper(lambda *a, **kw: foo(*a, **kw), foo)

It’s a weird gotcha, but engineering a workaround without introducing performance penalties is likely difficult (if not impossible). Perhaps the best option is just to document it and move on.

[robertwb]

scoder changed description from

In Python, one can do

class A:
    pass

def foo(self):
    print self

A.foo = foo
print A().foo()

and foo will get bound as expected. However, Cython creates PyCFunctions which do not properly bind when accessed (possibly by design). This is the root issue behind http://trac.cython.org/ticket/478, and will become more evident once closures start getting used.

PyCFunction does not allow subclassing. The only reasonable fix I see is to create a new type that wraps PyCFunction but behaves like PyFunction. This will have indirection overhead. To avoid overhead, one alternative is for our new type to have the same struct and slots as PyCFunction (with the exception of tp_name, tp_descr_get, tp_alloc, and tp_dealloc of course). This still may have overhead as the Python compiler optimizes for PyCFunction calls–perhaps the user should be allowed to control this (as part of an “ultra pure, I don’t care what cost” mode…)?

to

In Python, one can do

class A:
    pass

def foo(self):
    print self

A.foo = foo
print A().foo()

and foo will get bound as expected. However, Cython creates PyCFunctions which do not properly bind when accessed (possibly by design). This is the root issue behind http://trac.cython.org/ticket/478, and will become more evident once closures start getting used.

PyCFunction does not allow subclassing. The only reasonable fix I see is to create a new type that wraps PyCFunction but behaves like PyFunction. This will have indirection overhead. To avoid overhead, one alternative is for our new type to have the same struct and slots as PyCFunction (with the exception of tp_name, tp_descr_get, tp_alloc, and tp_dealloc of course). This still may have overhead as the Python compiler optimizes for PyCFunction calls–perhaps the user should be allowed to control this (as part of an “ultra pure, I don’t care what cost” mode…)? resolution to fixed status from new to closed commented

This works with the “binding=True” directive, which is now (since 0.17 IIRC) on by default for .py files.