Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Wrong input artifacts path in yaml compiled by python SDK

See original GitHub issue

/kind bug

What steps did you take and what happened:

Input artifacts path is wrong when input artifacts are from 2 different parent tasks.

What did you expect to happen:

Input artifacts path should be in correct format, and be accessible.

Additional information: [Miscellaneous information that will assist in solving the issue.]

get-subdirectory is the common parent task for list-items and list-items-2. I suppose the input artifact path of get-subdirectory should be same for both downstream tasks, but they are different.

path of list-items: $(workspaces.list-items.path)/get-subdirectory-Subdir image

path of list-items-2: /tmp/inputs/Directory1/data image

And list-items-2 cannot get the input artifacts from the path, and the error log is ls: /tmp/inputs/Directory1/data: No such file or directory


Here is the codes to generate the pipeline

filesystem_component_root = 'components/filesystem'
get_subdir_op = kfp.components.load_component_from_file(os.path.join(filesystem_component_root, 'get_subdirectory/component.yaml'))
list_item_op_1 = kfp.components.load_component_from_file(os.path.join(filesystem_component_root, 'list_items/component.yaml'))
list_item_op_2 = kfp.components.load_component_from_file('jizg/component.yaml')

def test_pipeline():
    def produce_dir_with_files_python_op(output_dir_path: OutputPath(), num_files: int = 10):
        import os
        os.makedirs(os.path.join(output_dir_path, 'subdir'), exist_ok=True)
        for i in range(num_files):
            file_path = os.path.join(output_dir_path, 'subdir', str(i) + '.txt')
            with open(file_path, 'w') as f:

    produce_dir_python_task = produce_dir_with_files_python_op(num_files=15)

    get_subdir_task = get_subdir_op(
        # Input name "Input 1" is converted to pythonic parameter name "input_1"

    list_items_task_1 = list_item_op_1(
        # Input name "Input 1" is converted to pythonic parameter name "input_1"

    list_items_task_2 = list_item_op_2(
        # Input name "Input 1" is converted to pythonic parameter name "input_1"


name: List items 2
description: Recursively list directory contents.
- {name: Directory1, type: Directory}
- {name: Directory2, type: Directory}
- {name: Items}
    image: alpine
    - sh
    - -ex
    - -c
    - |
      mkdir -p "$(dirname "$2")"
      ls -A -R "$0" > "$2"
      ls -A -R "$1" >> "$2"
    - inputPath: Directory1
    - inputPath: Directory2
    - outputPath: Items


  • Python Version (use python --version): 3.7.8
  • SDK Version: 0.3.0
  • Tekton Version (use tkn version): not installed
  • Kubernetes Version (use kubectl version): 1.16.15
  • OS (e.g. from /etc/os-release): Ubuntu 18.04

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

issue-label-bot[bot]commented, Oct 22, 2020

Issue-Label Bot is automatically applying the labels:

Label Probability
area/front-end 0.75

Please mark this comment with 👍 or 👎 to give our bot feedback! Links: app homepage, dashboard and code for this bot.

fenglixacommented, Oct 28, 2020

This is a bug of big data passing, as Tommy mentioned, will fix it.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Publish and download build artifacts - Azure Pipelines
Artifacts can be published at any stage of your pipeline. You can use YAML or the classic Azure DevOps editor to publish your...
Read more >
Error: No package found with specified pattern: D:\a\r1\a***.zip
It looks like in your artifacts there is not zip file and the task in the release need a zip file. ... @ShaykiAbramczyk...
Read more >
Build specification reference for CodeBuild
This topic provides important reference information about build specification (buildspec) files. A buildspec is a collection of build commands and related ...
Read more >
`.gitlab-ci.yml` keyword reference - GitLab Docs
include:project : The full GitLab project path. include:file A full ... Job artifacts are only collected for successful jobs by default, ...
Read more >
Create and Configure Jobs and Pipelines Using YAML
REST API to Access YAML Files · Validate a Job's or Pipeline's Configuration · Create a Job or a Pipeline Without Committing the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found