question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Should the compiler generate Pipeline or PipelineRun

See original GitHub issue

This is a general discussion, not a work item.

Conceptually a Pipeline is a static specification orchestrating how a set of Tasks should be executed. A PipelineRun is an instantiation of a pipeline. A pipeline is a re-usable artifact that can be “run” repeatedly, a pipeline-run is one-and-done.

Currently the KFP-Tekton compiler generates a YAML with (multiple) Tasks(s) and one Pipeline document. There are however certain features supported by the KFP DSL which are not easily transferred into an equivalent feature in Tekton Pipelines, but instead require the concept of a PipelineRun, like mounting PVC volume to a pipeline workspace or setting a timeout at the pipeline level.

Linked issues

  • Pipeline level timeout #56
  • Passing parameters between tasks #41
  • Volume mounts #51
  • Generate PipelineRun with embedded pipelineSpec and taskSpec #143

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:21 (18 by maintainers)

github_iconTop GitHub Comments

1reaction
ckadnercommented, Mar 26, 2020

PipelineRun with Pipeline definition embed

I think @Tomcli 's approach (PR #62) with generating an optional PipelineRun in addition to the existing Pipeline with Tasks will be the best compromise and allows to keep existing code intact. At least for the time being.

0reactions
k8s-ci-robotcommented, Jun 26, 2020

@Tomcli: Closing this issue.

In response to this:

Thanks @Udiknedormin /close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Tekton Pipelines
A Pipeline requires PipelineResources to provide inputs and store outputs for the Tasks that comprise it. You can declare those in the resources...
Read more >
Able to start pipelinerun from pipelinerun spec without ...
The pipelinerun does not create any taskruns/pods . When I do kubectl apply -f to a pipelinerun spec which does not provide volume...
Read more >
CI/CD pipelines - GitLab Docs
Pipelines can be manually executed, with predefined or manually-specified variables. You might do this if the results of a pipeline (for example, a...
Read more >
Build a pipeline | Vertex AI | Google Cloud
Compiler . You create a pipeline run on Vertex AI Pipelines using the Vertex AI Python client. When you run a pipeline, you...
Read more >
Build a Pipeline | Kubeflow
A tutorial on building pipelines to orchestrate your ML workflow. ... Not having this artifact available could make it harder to debug this ......
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found