Redesign

I’ve been kicking around some ideas for a different “calling Julia from Python” interface. While it may not replace pyjulia, this seems like a good venue to start the discussion.

The core of the idea is that a user would declare their the Julia functions they want exported to Python in a Julia file. e.g. they could have a file foo.jl containing

using PyCall

@pyexport sin(x::Float64)

Then on the python side pyjulia would provide hook into the Python importlib system, so that a user could just write:

import julia
import foo # would look up foo.jl if no foo.py exists
foo.sin(1.0)

Essentially, pyjulia would only be responsible for the importlib hooks and initialising the Julia kernel.

This is a little less “dynamic” than the current pyjulia approach, but I think there are some advantages

• it can be faster since the callbacks can make use of type inference on the arguments
• it should hopefully reduce segfaults by clearly defining an interface and restricting the types which are transferred back and forth
• it should avoid the need for calling the Julia eval from Python, since you can just add that code to the Julia file.
• it might make it possible to do static compilation: e.g. you would be able to compile the julia script into one dylib which Python just sees as an ordinary extension module (so you wouldn’t need the import julia at the start).
• Julia packages could define their Python interface in their own repo

Obviously, there is still a lot to be determined, e.g. how the @pyexport would handle conversions, but I wanted to elicit some feedback on the idea.

cc: @ViralBShah @StefanKarpinski @rejuvyesh @stevengj

[stevengj]

The downside is that Julia packages would have to define a Python interface. For me, the whole point of pyjulia and PyCall is for Julia and Python packages to be able to call one another without writing any “glue” code beforehand.

That being said, I think we should make it possible to define a “nicer” Python API by writing some glue code on top of pyjulia/PyCall, in the same way that something like SymPy.jl provides a nicer Julia API on top of PyCall to sympy.

So, whatever “static” glue code is supported here should be in addition to pyjulia in my opinion, not replacing it.

[StefanKarpinski]

One thought: might it be possible to make it look like a Python julia module has as many submodules as their are Julia packages with declared Python interfaces? So you’d write this instead:

import julia.foo as jlfoo
import foo as pyfoo

pyfoo.frizzle(x)
jlfoo.frob(y)

Benefits:

• makes it unambiguous that you want the Julia foo package, not a Python one by that name.
• avoids needing import julia just for the side-effect of starting the Julia interpreter.
• calling code could remain the same between the dynamic and static versions.