You can't manually enforce the order of local and plugin-based hooks

Description & Context

Some hooks come from plugins (like kedro-mlflow), whereas some may come from local hooks (like those). Currently, if many plugins and local hooks implement e.g. after_pipeline_run, there is no way to manually enforce order in which all those functions will be invoked.

This causes problems such as this one.

Also, while writing this post I realized another problem, which is using the name “hook” for both hook functions (e.g. after_pipeline_run hook function) and hook classes (e.g. DataCatalogHook class)

Steps to Reproduce

Available in the linked post above.

Expected Result

I assume we should be able enforce the order of hooks, assign them some priority or rules.

Your Environment

• kedro 0.18.3
• Python 3.9.14

[foxale]

Awesome. I guess that solves most cases (including mine), with the exception of those that require different orders for different hook functions.

[noklam]

@foxale I think this is supported already, although we don’t have docs to spill this out clearly.

• In general, I think the order should not matter, they should operate independently. Auto-registered hooks shouldn’t need to be aware of the existence of other plugins - this would be the common case.
• In special cases you need the fine grain control of order - you can import these hooks into your project directly and disable the automatic one.

# settings.py
from kedro_mlflow.hooks import MLflowHooks  # I make these import up but you get the point
DISABLE_HOOKS_FOR_PLUGINS = ("kedro-mlflow",)
HOOKS = (MLflowHook(), YourHook(),)  # This is always a LIFO queue - in fact the LIFO order is inherit from `pytest` already

One thing we may consider is to do the auto-register hook before the project hooks, but this is up for discussion.