Confusing error message when default pickling for intermediate storage fails

repro:

from dagster import (
    DagsterInstance,
    Output,
    Nothing,
    execute_pipeline,
    pipeline,
    reconstructable,
    solid,
)


@solid
def nothing_solid(_):
    yield Output(Nothing)


@pipeline
def repro():
    nothing_solid()


if __name__ == "__main__":
    execute_pipeline(
        reconstructable(repro),
        run_config={"execution": {"multiprocess": {}}, "storage": {"filesystem": {}}},
        instance=DagsterInstance.get(),
    )

[catherinewu]

another user reported getting “OSError: [Errno 22] Invalid argument” from the same line as above stack trace since the user’s output was several GB. It was not clear that the resolution was to configure a different intermediate storage

[yuhan]

Introduced a new error type DagsterObjectStoreError to capture errors during intermediate object store get and set.

updated the error message to show more details: