Run `numba` compilation as a post-install step?

Currently, this is the import performance of sgkit on a fresh install:

(testenv) benj@benj-X580VD:~$ time python -c "import sgkit"

real	0m16.715s
user	0m16.149s
sys	0m0.987s
(testenv) benj@benj-X580VD:~$ time python -c "import sgkit"

real	0m2.184s
user	0m2.414s
sys	0m0.914s

It might be possible to move the 14s of numba compilation to a post-install step, hopefully on pip and conda.

[tomwhite]

I had another look at this. The numba compilation is actually from guvectorize, not jit, since the latter is lazy. I don’t think guvectorize can be made lazy since you have to specify the signatures up front.

I think the simplest way forward would be to move all the guvectorize functions to separate modules and then import them from the functions that need them. This works since none of the guvectorize functions are in the public API.

So for popgen.py, for example, we could move a function like _divergence which is decorated with numba_guvectorize to popgen_accelerate.py (or similar), and then the public divergence function would import _divergence in the function body. Compilation would happen the first time divergence is called (not imported).

[tomwhite]

Another thing to try might be lazy compilation instead of the eager compilation (with types specified) that we use everywhere.