[RFC] loading packaged ray functions

Update: This is the google doc of the draft API: https://docs.google.com/document/u/1/d/1gLGl_eNUsf0O2se4rKg7fbf9IhHtst1w8ydGPNCXnQI/edit

This issue proposes a way to package and load functions defined in separate codebases, similar to how you can import packages in Golang: https://thenewstack.io/understanding-golang-packages/

A few use case examples include:

• Loading different model versions in a single Ray cluster for serving
• Importing an external Ray library released on GitHub that has different package dependencies
• Composing a pipeline from different published “ML operators”

It depends on the runtime env RFC: https://github.com/ray-project/ray/issues/14019 And is related to the ML pipeline operator RFC: https://github.com/ray-project/ray/issues/14078

Proposed user API:

# Load from local
> my_pkg = ray.util.load_package(path = "~/path/to/my_pkg.yaml")

# Load from git
> my_pkg = ray.util.load_package(
   git_url = "https://github.com/my-project/my-repo.git",
   git_refspec = "release/1.2.0",
   path = "path/to/my_pkg.yaml")

> print(my_pkg.runtime_env())
{"py_requirements": ["pytorch==1.6", "opencensus"], "files": FilesInGitRepo(...)}

# Run remote functions like normal
> my_pkg.my_func.remote(1, 2)

# Create actors like normal
> actor = my_pkg.MyActor.remote(3, 4)

# Create new remote funcs in a package
> @ray.remote(runtime_env=my_pkg.runtime_env())
def f(): ...

This RFC doesn’t define the implementation of the packaging format, but there are a few open questions there:

• Do we need to keep a separate YAML config for packages? If we don’t, we might need to import the package at load time, which could be problematic since the runtime_env cannot be loaded before the import.
• Do we need to declare the function/class signatures in the YAML config?
• How will this work with cross-language packages?
• How do we handle serialization issues across different runtime environments?

[ericl]

To clarify, it’s the same as a remote actor definition.

@ray.remote
class MyActor:
   pass

type(MyActor)  # ray.actor.ActorClass
a = MyActor.remote()

pkg = ray.util.load_package("...")
type(pkg.MyActor)   # ray.actor.ActorClass
a = pkg.MyActor.remote()

The same goes for the remote function definitions.

when loading a package, is a detached actor created in the backend? and this my_pkg.myactor is just the handle of that remote actor?

Nothing is actually created, just the remote actor and function definitions are available.

[valiantljk]

what does ‘remote actor stub’ mean here? when loading a package, is a detached actor created in the backend? and this my_pkg.myactor is just the handle of that remote actor?