parfor and brmesolve - Cythonizing problem in v5

Bug Description

When combining parfor with brmesolve and a time-dependent Hamiltonian, the compilation of the Hamiltonian seems to interfer with the correct functioning of parfor. The call results in an error, that the coefficient file is too short.

I guess this could also happen when using parfor with other solvers.

Code to Reproduce the Bug

from qutip import brmesolve, sigmaz, sigmax, basis, parfor
import numpy as np

psi0 = basis(2, 0)
H_S = np.pi * sigmaz()
H_I = sigmax()
tlist = np.linspace(0, 100, 101)
pulse_shape = "exp(- t**2)"

def brme(A):
    H_tmp = [A * H_S, [H_I, pulse_shape]]

    return brmesolve(
        H_tmp, psi0, tlist, e_ops=[sigmaz()]
    ).expect[0][-1]

results = parfor(brme, np.linspace(0, np.pi, 10))

Code Output

ImportError: /home/chris/.qutip/qutip_coeffs_1.1/qtcoeff_4c859672674b8b603bd243ba3f4a8d.cpython-38-x86_64-linux-gnu.so: file too short

Expected Behaviour

parfor should wait for the compilation to finish and then start the parallel processes.

Your Environment

QuTiP Version:      5.0.0.dev0+c64f38e
Numpy Version:      1.23.0
Scipy Version:      1.8.1
Cython Version:     0.29.30
Matplotlib Version: 3.5.2
Python Version:     3.8.10
Number of CPUs:     4

Additional Context

No response

[hodgestar]

@christian512 Could you remove the “pre-compilation” of the Cython files from qutip/qutip-tutorials#31? Then if that works we can close this issue.

[christian512]

I removed the “pre-compilation” and it worked in the tutorial. Since we will change from using brmesolve to BRSolver in that notebook (see https://github.com/qutip/qutip-tutorials/pull/31#discussion_r946862389) I will close this issue.