question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[sdk] Task.after() on Condition OpsGroup vs ContainerOp

See original GitHub issue

We have few questions on how the condition is constructed in Argo. e.g. using the below condition example https://github.com/kubeflow/pipelines/blob/master/samples/core/condition/condition.py

When we try to define a containerOp that run after dsl.condition or containerOp inside the condition block, it creates the same result when compiles to Argo. We are wondering is it the expected DSL behavior where the task dependencies should point to the Argo condition task in both cases?

@dsl.pipeline(
    name='Conditional execution pipeline',
    description='Shows how to use dsl.Condition().'
)
def flipcoin_pipeline():
    flip = flip_coin_op()
    with dsl.Condition(flip.output == 'heads') as cond_a:
        random_num_head = random_num_op(0, 9)
        random_num_head2 = random_num_op(0, 9)

    with dsl.Condition(flip.output == 'tails') as cond_b:
        random_num_tail = random_num_op(10, 19)
        random_num_tail2 = random_num_op(10, 19)

    # print_output and print_output2 produce the same dependencies
    print_output = print_op('after cond_a cond_b').after(cond_a).after(cond_b)
    print_output2 = print_op('after random_num_head random_num_tail').after(random_num_head).after(random_num_tail)

In Argo dag:

    dag:
      tasks:
      - name: condition-1
        template: condition-1
        when: '"{{tasks.flip-coin.outputs.parameters.flip-coin-output}}" == "heads"'
        dependencies: [flip-coin]
      - name: condition-2
        template: condition-2
        when: '"{{tasks.flip-coin.outputs.parameters.flip-coin-output}}" == "tails"'
        dependencies: [flip-coin]
      - {name: flip-coin, template: flip-coin}
      - name: print
        template: print
        dependencies: [condition-1, condition-2]
      - name: print-2
        template: print-2
        dependencies: [condition-1, condition-2]

Thanks.

related docs: https://docs.google.com/document/d/1QPWKoeiPFDcI1JWH-nMe7x_xIJekL11bhu7Fp3fNT24/edit#

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:9 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
Tomclicommented, Apr 12, 2021

Thanks @Ark-kun. Good to know this is a bug in the KFP Argo compiler. We are able to work around on the KFP Tekton compiler because we decided not to put conditional tasks into a sub-dag to optimize it for Tekton.

We want to make sure KFP Argo also compiles dependency on conditional task, so we are not diverging the runtime behavior with the same KFP DSL.

0reactions
stale[bot]commented, Apr 18, 2022

This issue has been automatically closed because it has not had recent activity. Please comment “/reopen” to reopen it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

kfp.dsl package — Kubeflow Pipelines documentation
Configures the op_transformers which will be applied to all ops in the pipeline. The ops can be ResourceOp, VolumeOp, or ContainerOp. Parameters: transformer...
Read more >
Introduction to the Pipelines SDK - Kubeflow
The Kubeflow Pipelines SDK provides a set of Python packages that you can use to specify and run your machine learning (ML) workflows....
Read more >
Chapter 4. Kubeflow Pipelines - O'Reilly
An SDK for defining, building, and deploying pipelines in Python. Notebook support for using the SDK and pipeline execution.
Read more >
Source - GitHub
Condition ([\#8182](https://github.com/kubeflow/pipelines/issues/8182)) ... task dependency referencing tasks inside a sibling condition or loop group.
Read more >
kubeflow/pipelines release history - changelogs.md
Recent releases and changes to kubeflow/pipelines. ... Block task dependency referencing tasks inside a sibling condition or loop group.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found