Auto-generation of wrappers for C structs/C++ classes

cdef extern gives Cython access to existing C/C++ libraries. To provide Python-level access generally requires the user to write a wrapper in Cython. For functions and enums Cython can automatically do this by defining them as cpdef. For C structs and C++ classes that feature doesn’t currently exist (although Cython does generate a C struct <-> dict conversion).

Cython has most of the information to generate a basic wrapper - it know what attributes and functions the type has and whether they’re convertable to a Python representation. Therefore it seems reasonable for Cython to create this wrapper rather than forcing the user to do it (which is largely just duplicating the existing definition in an extension class, forwarding to self.obj).

This issue suggests adding that feature.

Link to cython-devel thread.

A few notes:

• Syntax is up for debate. Some suggestions
  • cython.autowrap[c_type, **keyword_options] - probably rejected because the square brackets make it less easily translated into pure-python mode.
  • cython.autowrap(c_type, **keyword_options)
  • @cclass decorator on struct/c++ class. Disadvantage is that it’d hide the name of the underlying C type (which would be inconvenient for writing Cython code around it). One option might be that @cython.cclass hides the name but it could also be used as a call to give a different name, e.g. ext_class_wrapper = cython.cclass(c_type) ctypedef cython.cclass(c_type) ext_class_wrapper
  • cpdef for consistency with automatic wrapping of functions/structs. Disadvantage is that it hides the name and there isn’t an easy workaround.
• It isn’t important to handle everything. The user can always inherit from the created extension type to define anything that’s missed.
• How to handle C++ functions that return references is always tricky (because the chances are something owns it, but who knows what). I’d propose not to handle this, and only deal with “return by value”.
• I’m not proposing to use these types for automatic coercion between C/C++ objects and Python objects. It’d be an “opt-in” feature.
  • partly not to break existing struct<->dict conversion and vector<->list (etc.) conversions.
  • partly because it’s probably possible to end up with the generated wrappers not compiling in C++ occasionally, and having them opt-in gives the user a chance of fixing that.

[scoder]

IIUC, autowrap uses Cython’s own parse tree for the generation, and then generates Cython code for the result.

That should make it possible to turn it into a tree transformation, generate Cython utility code, and then merge that into the current module. Tree transforms represent the different steps in the compiler pipeline, and the rest is a similar process to the way the Python wrapper for memory views is currently implemented.

It seems ok to leave it as an external tool for now, but I agree that it could tie more into the way Cython works internally. It could just become an optional dependency.

[poshul]

@da-woods I represent OpenMS, who have just taken over the development of Autowrap. We would be very happy to see this tool integrated more deeply into Cython.