[dagit] Dagit does not run on M1 (Apple Silicon)

Summary

I was going through the Getting Started docs for Dagster and was following along on a newer Macbook using an M1 chip. When I tried to start Dagit, I ran into an error, and I’m not able to figure out a quick workaround.

I think the issue might be with an upstream dependency, but I wanted to report it here first, in case I was mistaken.

Reproduction

1. Install Dagster with pip install dagster.
2. Create hello_world.py as described in the getting started docs.
3. Install Dagit with pip install dagit.
4. Run dagit -f hello_world.py

Expected behavior

The Dagit command does not output an error, and localhost:3000 hosts an instance of the Dagit UI.

Actual behavior

The Dagit command fails with this error:

Traceback (most recent call last):
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/bin/dagit", line 5, in <module>
    from dagit.cli import main
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagit/__init__.py", line 1, in <module>
    from dagster.core.utils import check_dagster_package_version
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagster/__init__.py", line 175, in <module>
    from dagster.core.launcher import DefaultRunLauncher
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagster/core/launcher/__init__.py", line 1, in <module>
    from .base import (
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagster/core/launcher/base.py", line 8, in <module>
    from dagster.core.workspace.workspace import IWorkspace
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagster/core/workspace/__init__.py", line 1, in <module>
    from .context import IWorkspaceProcessContext, WorkspaceProcessContext
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagster/core/workspace/context.py", line 13, in <module>
    from dagster.core.host_representation import (
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagster/core/host_representation/__init__.py", line 9, in <module>
    from .external import (
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagster/core/host_representation/external.py", line 24, in <module>
    from .handle import JobHandle, PartitionSetHandle, PipelineHandle, RepositoryHandle
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagster/core/host_representation/handle.py", line 4, in <module>
    from dagster.core.host_representation.origin import ExternalRepositoryOrigin
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/dagster/core/host_representation/origin.py", line 8, in <module>
    import grpc
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/grpc/_compression.py", line 15, in <module>
    from grpc._cython import cygrpc
ImportError: dlopen(/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so, 0x0002): tried: '/Users/praneetsahgal/.pyenv/versions/3.10.0/lib/python3.10/site-packages/grpc/_cython/cygrpc.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cygrpc.cpython-310-darwin.so' (no such file), '/usr/lib/cygrpc.cpython-310-darwin.so' (no such file)

Additional Info about Your Environment

• Device: 2021 16-inch Macbook Pro (running an M1 chip)
• Python version: 3.10.0
• Pip version: 21.3.1
• I was prompted to install Rosetta earlier, so I think I have it installed.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We factor engagement into prioritization.

[matheus-rossi]

Using python 3.9.x and installing grpc with conda, solved the problem to me

https://github.com/grpc/grpc/issues/25082#issuecomment-878271247

conda install grpcio

[sspaeti]

OK, I finally solved the issue for me. I spend hours on it and as I installed it forth and back it’s hard to say what in the end solved the problem. But still, I want to give some pointers to folks who have similar issues. My problem was deeper that asdf was probably having a wonky python installation or that this installation wasn’t properly set up with arch=arm64/

I believe at the end solved it by completely removing asdf as explained here. And reinstall with these additional settings:

brew update && upgrade
brew install llvm
echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc
export PYTHON_BUILD_HOMEBREW_OPENSSL_FORMULA=openssl@3

Now I could properly install any python version without errors such as

asdf install python 3.9.15
asdf global python 3.9.15

I could then successfully install the problematic grpcio and also run dagster/dagit without below error that I got the strange error in the meantime.

rm -r ~/.venvs/dagster-test && python -m venv ~/.venvs/dagster-test && source ~/.venvs//dagster-test/bin/activate && pip install --upgrade pip && pip install dagster dagit

The error I got after I managed to install grpcio when starting dagit or dagster:

importerror: dlopen(/users/sspaeti/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so, 0x0002): tried: '/users/sspaeti/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/system/volumes/preboot/cryptexes/os/users/sspaeti/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so' (no such file), '/users/sspaeti/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))