Support specifying a conda/virtual environment to use in the `LocalRun` run-config

When deploying a flow locally using a LocalRun run-config, it would be useful to support running a flow using a different python environment than the one the agent is currently running. This would let users create and deploy flows using isolated environments (i.e. a conda environment per flow).

The types of environments we’d like to support:

• A conda environment, specified either by name or absolute path
• A virtual environment, specified by absolute path
• A python interpreter to use, specified by name on PATH (e.g. python3.8) or full path to interpreter

We could do this with either:

• A single kwarg (python_env?) that takes in a URI where the scheme specifies the type of environment. This is also used by dask-yarn, but may be non-intuitive. Examples:

  • python_env="conda://my-env": a conda environment by name
  • python_env="conda:///full/path/to/env": a conda env by path
  • python_env="venv:///full/path/to/venv": a virtual environment by path
  • python_env="python:///full/path/to/python": full path to a python interpreter

• One kwarg for each type. Specifying multiple kwargs is an error (can’t have both a conda and venv). Examples:

  • conda_env="my-env": a conda environment by name
  • conda_env="/full/path/to/env": a conda environment by full path
  • virtual_env="/full/path/to/env": a virtual environment by full path
  • python_env="/full/path/to/python": full path to a python interpreter
  • python_env="python3.8": python interpreter command name

I’m leaning towards the second option, as it seems clearer to me.

[madkinsz]

We’re implementing this in Orion, so I’ll close this for now.

[madkinsz]

Orion was our code name for the 2.0 version. This feature is included in 2.0a7. It won’t be included in 1.0.