Pipeline node output artifacts should not be overridden if a node is executed multiple times

Scenario:

• Create a notebook pipeline.
• Create a notebook (abc.iynb) that produces a unique result during each execution.
• Add the same [notebook] node twice to the pipeline.
• Run the pipeline.
• Inspect the output artifacts abc.ipynb and abc.html

The outputs of notebook execution 1 are overridden by the outputs of notebook execution 2.

[ptitzler]

The overwrite doesn’t occur when the node is processed, but rather when the outputs are uploaded to COS

Yes, that’s what I was trying to raise.

[MLnick]

@ptitzler yes, hyperparameter tuning will generally have the same input data, with the only difference in runs being the input parameters to the ML pipeline / model. The results (trained model) would generally be different.

And yes also random init of models will lead to slightly different results each training run (unless a fixed random seed is used, which is not uncommon for reproducibility of pipelines).

But yes this would cover any case where the output of the node depends on the input parameters / env variables.