Distributed processes not finding modules

I have a cluster of PCs on the same network and reading from the same mounted python directory.

I have a layout of code like this:

Project\
    package\
        data\
        __init__.py
        fitting.py
        code2.py
        etc...
output\
script.py

The workers are started in the same mounted directory as the project above. However, when I run script.py, the worker spews out a load of unicode nonsense ,then the traceback below and then the worker apparently crashes. Apparently the module isn’t there

Traceback (most recent call last):
  File "E:/Dropbox/Project/script.py", line 14, in <module>
    fitter.save_results('output')
  File "E:\Dropbox\Project\Package\fitting.py", line 410, in save_results
    r = future.result()
  File "C:\Anaconda\lib\site-packages\distributed\executor.py", line 96, in result
    six.reraise(*result)
  File "C:\Anaconda\lib\site-packages\distributed\core.py", line 68, in loads
    return pickle.loads(x)
ImportError: No module named package.fitting

script.py:

from distributed import Executor
from package import fitting
executor = Executor('127.0.0.1:8786')
T = prepare_data()  # pandas dataframe
fitter = fitting.Fitter()  # my fitting object
fitter.fit_mulitple(T, executor)  # make futures in object
fitter.save_results()  # save to disk on local machine

fitting.py:

class Fitter(object):
    ....

    def fit_multiple(data_frame, executor):
        self.futures = [executor.submit(self.fit, series, append=False) for _, series in data_frame.iterrows()]  # submit the `fit` function for each row (`fit` is pure when append=False but requires other modules in the package)

    def save_results():
        return [saving_function(f.result()) for f in self.futures]  # generic save to hdf5 

[rahatzamancse]

I am having the same issue, and I have run dask-scheduler and dask-workers in local machine to test. From the same directory, if the program is launched, it goes smoothly. But ask workers do not seem to find the module of another python file in the same directory.

[mrocklin]

Your workers don’t seem to be able to find the package.fitting package. Can you verify that a normal Python session started in the same place is able to find that package? Perhaps you can install it or put it on the PYTHONPATH to help your workers out?

The workers are just Python processes. They aren’t doing anything special with paths or your software environment. It’s mostly up to you to ensure that all of your workers have the same software environment.

How are you setting up your workers? If it is with ssh then you should beware that sometimes actions in .bashrc or wherever you set up your user environment sometimes don’t persist to ssh sessions.