[sdk] Task.after() on Condition OpsGroup vs ContainerOp
See original GitHub issueWe have few questions on how the condition is constructed in Argo. e.g. using the below condition example https://github.com/kubeflow/pipelines/blob/master/samples/core/condition/condition.py
When we try to define a containerOp that run after dsl.condition or containerOp inside the condition block, it creates the same result when compiles to Argo. We are wondering is it the expected DSL behavior where the task dependencies should point to the Argo condition task in both cases?
@dsl.pipeline(
name='Conditional execution pipeline',
description='Shows how to use dsl.Condition().'
)
def flipcoin_pipeline():
flip = flip_coin_op()
with dsl.Condition(flip.output == 'heads') as cond_a:
random_num_head = random_num_op(0, 9)
random_num_head2 = random_num_op(0, 9)
with dsl.Condition(flip.output == 'tails') as cond_b:
random_num_tail = random_num_op(10, 19)
random_num_tail2 = random_num_op(10, 19)
# print_output and print_output2 produce the same dependencies
print_output = print_op('after cond_a cond_b').after(cond_a).after(cond_b)
print_output2 = print_op('after random_num_head random_num_tail').after(random_num_head).after(random_num_tail)
In Argo dag:
dag:
tasks:
- name: condition-1
template: condition-1
when: '"{{tasks.flip-coin.outputs.parameters.flip-coin-output}}" == "heads"'
dependencies: [flip-coin]
- name: condition-2
template: condition-2
when: '"{{tasks.flip-coin.outputs.parameters.flip-coin-output}}" == "tails"'
dependencies: [flip-coin]
- {name: flip-coin, template: flip-coin}
- name: print
template: print
dependencies: [condition-1, condition-2]
- name: print-2
template: print-2
dependencies: [condition-1, condition-2]
Thanks.
related docs: https://docs.google.com/document/d/1QPWKoeiPFDcI1JWH-nMe7x_xIJekL11bhu7Fp3fNT24/edit#
Issue Analytics
- State:
- Created 2 years ago
- Comments:9 (7 by maintainers)
Top Results From Across the Web
kfp.dsl package — Kubeflow Pipelines documentation
Configures the op_transformers which will be applied to all ops in the pipeline. The ops can be ResourceOp, VolumeOp, or ContainerOp. Parameters: transformer...
Read more >Introduction to the Pipelines SDK - Kubeflow
The Kubeflow Pipelines SDK provides a set of Python packages that you can use to specify and run your machine learning (ML) workflows....
Read more >Chapter 4. Kubeflow Pipelines - O'Reilly
An SDK for defining, building, and deploying pipelines in Python. Notebook support for using the SDK and pipeline execution.
Read more >Source - GitHub
Condition ([\#8182](https://github.com/kubeflow/pipelines/issues/8182)) ... task dependency referencing tasks inside a sibling condition or loop group.
Read more >kubeflow/pipelines release history - changelogs.md
Recent releases and changes to kubeflow/pipelines. ... Block task dependency referencing tasks inside a sibling condition or loop group.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Thanks @Ark-kun. Good to know this is a bug in the KFP Argo compiler. We are able to work around on the KFP Tekton compiler because we decided not to put conditional tasks into a sub-dag to optimize it for Tekton.
We want to make sure KFP Argo also compiles dependency on conditional task, so we are not diverging the runtime behavior with the same KFP DSL.
This issue has been automatically closed because it has not had recent activity. Please comment “/reopen” to reopen it.