question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

allow_reuse to force run steps in Azure ml pipeline python sdk v2

See original GitHub issue
  • Package Name: azure.ai.ml
  • Package Version:
  • Operating System: linux
  • Python Version: 3.9

Describe the bug I am running a training pipeline on azure ml python sdk v2. It is running fine but when I am reinitiating it, it takes only 1 second to finish which I assume is taking previous step from last ran pipeline. In v1 sdk we have allow_reuse parameter to stop this behaviour. But how can we do that in ml python sdk v2.

I used the following code to create a step:

 data_prep_component = command(
     name="data_prep_credit_defaults",
     display_name="Data preparation for training",
     description="Preprocessing the training data",
     inputs={
         #"input_dir": Input(type="string"),
         "output_dir": Input(type="string"),
     },
     outputs={
         #"output_dir": Output(type="string"),
     },
     # The source folder of the component
     code=PATH,
     command="""python datapreprocessor.py \
             --output_dir ${{inputs.output_dir}} \
             """,
     environment="bp_environment:3.0",
 )

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:8 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
wangchao1230commented, Jun 30, 2022

Thanks for the feedback. Please use is_deterministic argument.

command(is_deterministic=Fasle)
0reactions
msftbot[bot]commented, Jul 15, 2022

Hi @rjtshrm, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

allow_reuse in Azure ml sdk v2 - Microsoft Q&A
I am running a training pipeline on azure ml python sdk v2. ... to finish which I assume is taking previous step from...
Read more >
Tutorial: ML pipelines with Python SDK v2 - Azure
You'll learn how to use the AzureML Python SDK v2 to: Connect to your Azure ... You'll then run the pipeline, deploy the...
Read more >
Create and run ML pipelines - Azure Machine Learning
Create the resources required to run an ML pipeline: Set up a datastore used to access the data needed in the pipeline steps....
Read more >
Upgrade parallel run step to SDK v2 - Azure Machine Learning
Use parallel_run_function to create parallel job by replacing ParallelRunConfig and ParallelRunStep in v1. · Migrate your v1 pipeline to v2. Then ...
Read more >
Upgrade pipelines to SDK v2 - Azure Machine Learning
In this article. Run a pipeline; Mapping of key functionality in SDK v1 and SDK v2; Step and job/component type mapping; Related documents....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found