Changing launcher for InteractiveContext

I’m developing an internal launcher that contains some optimizations for internal usecases (and some instrumentation).

I’ve been able to successfuly make it run by changing the PipelineConfig(supported_launcher_classes=[MyLauncher]) when using the AirflowLauncher and BeamLauncher but it seems that it can’t be changed for InteractiveContext.

I know InteractiveContext is an experimental feature, but looking at the codebase it seems that there’s a reference to enable changing the launcher class.

Wondering if there’s any good way to do this at the moment (or plan to enable this in the near future) or if I should just reimplement InteractiveContext to overwrite the launcher.

[casassg]

We have an internal launcher that tracks components executions/failures across the company. Allows us (developers) to understand our internal usage (modelers, other engineers) better and track performance of components.

Another use case that I was thinking was to add integrations with internal services (data registration services and/or internal dashboards).

I believe launchers are the best place for this type of customization (better than having to instrument component executors one by one?), and so far we are only missing enabling this in the InteractiveContext.

I will enable it by overwriting the run method in an internal InteractiveContext.

[1025KB]

I see, I will keep this as a feature request for supporting different launcher in InteractiveContext