Loading scikit-learn pipeline with ray serve raises async_goal_exception

Search before asking

• I searched the issues and found no similar issues.

Ray Component

Ray serve

What happened + What you expected to happen

I am using a scikit-learn pipeline that is saved as a pickled file. When I load the pipeline again everything is fine and I can use it as expected. But I am not able to load the pickled file when using the @serve.deployment mechanism. I can open it with open() but as soon as I use pickle.load() it fails

@serve.deployment(num_replicas=1)
class Model():
  def __init__(self):
    pipe_file = "dump.pkl"
    assert os.path.exist(pipe_file), "File not found"

    with open(pipe_file,"rb") as f:
      self.pipe = pickle.load(f)

  def __call__(self):
    return self.pipe.transform(np.random.random(100,100))

Model.deploy()

It just fails raising async_goal_exception

Versions / Dependencies

Ray: 1.9.1 Python: 3.7.2 and also 3.9.7 OS: Ubuntu 18.04 LTS

Reproduction script

I created a very simple toy example that can be run in colab and creates the same error

https://colab.research.google.com/gist/markste-in/5a80cb9b5d7a71888c31f1c1d53c544e/ray_issue.ipynb

Anything else

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[shrekris-anyscale]

Thanks for the feedback– I added a Github issue about Serve exception handling, so we can track it. Good luck!

[markste-in]

Update: It seems I opened the file with open(file_path,“wb”) instead of open(file_path,“rb”).

That seems to throw async_goal_exception.

I am not sure if it would be possible to improve the error handling since “async_goal_exception” obfuscate the real issue.