question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add "origin" information to generated pipeline PY/DSL

See original GitHub issue
  • Create a basic pipeline (any pipeline will do)
  • Run the pipeline
  • Open the pipeline in Kubeflow Pipelines Note that version source and description are currently empty.

image

For the sake of traceability Elyra should (if possible) provide this information to enable Kubeflow Pipelines users to identify the pipeline origin. e.g Elyra <pipeline name>

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:9 (9 by maintainers)

github_iconTop GitHub Comments

1reaction
ptitzlercommented, Aug 4, 2020

I do believe having identifying information embedded is important for production-ish use cases and would therefore advocate for a staged approach

Motivation: in a managed production environment many pipelines might be running over time. If there’s a need for an ops person to reach out to the owner of a workflow (e.g. because it was/is be utilizing a large amount of resources in a cluster) it should be possible to access identifying pipeline information that associates a pipeline with the entity that submitted it.

0reactions
ptitzlercommented, Aug 4, 2020

Is ‘Description’ something you see coming from the end-user (which would likely be helpful)? If not, we could embed what I thought would be good for ‘Version source’ in the description, something like: ‘Elyra 1.0.3 pipeline’. But I think, assuming we can set ‘Description’, this particular value should come from the user and let ‘Version source’ be a “system-owned” value (meaning it derives from and is set by the server).

I haven’t been able to find any information about the version source. I did reach out to a few folks in Slack who might be able to point us in the right direction and am waiting for them to respond. I believe your proposal is good:

  • description = user provided (with a useful default if nothing is specified)
  • version source = Elyra provided (if possible; if not “version source” information could be the default for ^^)

I do believe having identifying information embedded is important for production-ish use cases and would therefore advocate for a staged approach:

  • Stage 1 (quick delivery): description = Elyra generated “static” text (doesn’t require any UI updates or changes to the pipeline metadata)
  • Stage 2: <something more comprehensive>
Read more comments on GitHub >

github_iconTop Results From Across the Web

Downstream pipelines - GitLab Docs
A downstream pipeline is any GitLab CI/CD pipeline triggered by another pipeline. Downstream pipelines run independently and concurrently to the upstream ...
Read more >
Tag your Git repo using Gitlab's CI/CD pipeline - Neron Joseph
Here, we are looking at methods of tagging using Gitlab's CI/CD pipeline code ... Then, we are removing the existing origin and add...
Read more >
awesome-python-1/README.md at master - GitHub
Aims to cover everything from data mining to deep learning. spotify/luigi - Luigi is a Python module that helps you build complex pipelines...
Read more >
Salt Documentation - manpages.ubuntu!
The topology is a simple server/client model with the needed functionality built into a single set of daemons. While the default configuration will...
Read more >
Salt Documentation
If confirmed working correctly, please report and add a note on this page with the ... Generate Keys and host record and start...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found