Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Missing env var for run storage config - error is pretty rough

See original GitHub issue

Found this when working on Helm chart.

dagster.yaml:

    run_storage:
      module: dagster_postgres.run_storage
      class: PostgresRunStorage
      config:
        postgres_db:
          username: {{ .Values.postgresql.postgresqlUsername}}
          password:
            env: DAGSTER_PG_PASSWORD
          hostname: {{ template "dagster.postgresql.fullname" . }}
          db_name:  {{ .Values.postgresql.postgresqlDatabase	}}
          port: {{ .Values.postgresql.service.port }}

Forgot to set DAGSTER_PG_PASSWORD on the Celery workers, then when initializing, they crash with error:

Traceback (most recent call last):
  File "/usr/local/bin/dagster-celery", line 11, in <module>
    load_entry_point('dagster-celery', 'console_scripts', 'dagster-celery')()
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/dagster-celery/dagster_celery/cli.py", line 159, in worker_start_command
    pythonpath = get_config_dir(config_yaml)
  File "/dagster-celery/dagster_celery/cli.py", line 54, in get_config_dir
    instance = DagsterInstance.get()
  File "/dagster/dagster/core/instance/__init__.py", line 184, in get
    return DagsterInstance.from_config(_dagster_home())
  File "/dagster/dagster/core/instance/__init__.py", line 206, in from_config
    return DagsterInstance.from_ref(instance_ref)
  File "/dagster/dagster/core/instance/__init__.py", line 214, in from_ref
    run_storage=instance_ref.run_storage,
  File "/dagster/dagster/core/instance/ref.py", line 176, in run_storage
    return self.run_storage_data.rehydrate()
  File "/dagster/dagster/core/serdes/__init__.py", line 205, in rehydrate
    config_dict,
  File "/dagster/dagster/core/errors.py", line 301, in __init__
    error_message = friendly_string_for_error(error)
  File "/dagster/dagster/config/errors.py", line 128, in friendly_string_for_error
    check.failed('{} (friendly message for this type not yet provided)'.format(error.reason))
  File "/dagster/dagster/check/__init__.py", line 109, in failed
    raise_with_traceback(CheckError('Failure condition: {desc}'.format(desc=desc)))
  File "/usr/local/lib/python3.7/site-packages/future/utils/__init__.py", line 446, in raise_with_traceback
    raise exc.with_traceback(traceback)
dagster.check.CheckError: Failure condition: DagsterEvaluationErrorReason.FAILED_POST_PROCESSING (friendly message for this type not yet provided)