Docker executor launches using virtualenv python

Summary

If I run Dagit using a virtulaenv, the Docker executor tries to run a command with my virtual environment’s python (which doesn’t exist in the Docker container)

Reproduction

Run dagit from within a virtualenv. Then create a config that looks something like this:

execution:
  config:
    image: <Docker Image>
...

And the launched command should fail since it will have something like this as its default command:

"Cmd": [
                "/Users/nlarusstone/.virtualenvs/pipeline/bin/python3",
                "-m",
                "dagster",
                "api",
                "execute_step",
               ....

Error will look like:

/opt/venv/bin/python3: can't open file '/Users/nlarusstone/.virtualenvs/pipeline/bin/python3': [Errno 2] No such file or directory

Additional Info about Your Environment

Dagster 0.14.20 OSX 11.6.2 Docker 20.10.11

Message from the maintainers:

Impacted by this bug? Give it a 👍. We factor engagement into prioritization.

[gibsondan]

https://github.com/dagster-io/dagster/pull/8571 will fix the first issue you ran into and is something we’ve been meaning to do for a while, thanks for bringing it to our attention

[gibsondan]

Hi @nlarusstone - sorry for the trouble here. I think we can clean this up so that the default entry point is just “dagster” in this case.

In the meantime, running your code in your own grpc server (https://docs.dagster.io/concepts/repositories-workspaces/workspaces#running-your-own-grpc-server) is one potential workaround that I believe would make your steps run just using ‘dagster’ as the entrypoint rather than a specific python environment.