Allow defining post-run hooks on asset jobs

What’s the use case?

As the define_asset_job() method doesn’t take a hooks argument, the only way to execute a post-run hook is by creating a RunStatusSensor.

In a production environment however, each run will typically be running in its own separated environment/pod/ECS task, effectively meaning the sensor can’t reliably fetch any information from the run that’s saved in the file system of the isolated environment, I’m thinking of e.g. uploading the executed SQL after running a dbt op, but I imagine there’s many more use cases.

Ideas of implementation

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[sryza]

A workaround for the timebeing could be to use the runtime_metadata_fn argument of load_assets_from_dbt_project. It lets you return metadata that gets attached to the asset materialization. You could also put code inside of it that writes to S3.

[askvinni]

Ah, that’s an interesting workaround, definitely fit for my immediate use case.