Visualising different environments on Kedro-Viz

Description

I am creating my kedro pipelines dynamically from a custom JSON file which is part of my configuration per environment. So in a fully configured environment I would have following files:

# ./conf/environment_name
- catalog.yml
- credentials.yml
- logging.yml
- parameters.yml
- pipelines.json # my custom pipelines that are generated dynamically

That means running pipelines for me is only possible using kedro run --env=environment_name

Running kedro viz would only work for me in the default local environment but would not allow me to visualize pipelines per environment.

Can someone highlight the way on how to run kedro viz for different environments? It’s not my number 1 issue at the moment but when the time comes I’d also be happy to submit a PR. Would it work if I somehow use the React component instead of kedro viz from the command line?

Context

I’d love this change because the kedro workflow I am trying to implement would contain several different experiments in the same repository with just different config files for each experiment. That way I could visualize different pipelines per repository.

Possible Implementation

Not sure how it works under the hood, but giving the same parameter --env=environment_name parameter to kedro and kedro viz would be cool I guess.

Checklist

• [ environments, configuration]

[921kiyo]

It is implemeted in https://github.com/quantumblacklabs/kedro-viz/pull/93 with --env flag, so I’m closing this 😃

[OFranke]

I will once I do but it might take some time since it’s an issue I might need to fix latest in 2 month or so.

Sure, what I am currently trying to build is a kedro project which contains nodes that can perform general data operations. The nodes itself are all functional and have no side-effects, so users should be able to stick together a pipeline via frontend (guess it will be React), upload them to a new kedro environment folder (e.g. experiment_1, experiment_2) and then run the pipelines per environment.

Then I wanted to use kedro-viz to display to the user what kind of pipeline he is putting together, maybe I can make this work in real-time somehow. I even thought about reversing the whole thing to actually generate the pipeline config via drag&drop based on kedro-viz somehow. But thats far in the future atm.