allow_reuse to force run steps in Azure ml pipeline python sdk v2
See original GitHub issue- Package Name: azure.ai.ml
- Package Version:
- Operating System: linux
- Python Version: 3.9
Describe the bug I am running a training pipeline on azure ml python sdk v2. It is running fine but when I am reinitiating it, it takes only 1 second to finish which I assume is taking previous step from last ran pipeline. In v1 sdk we have allow_reuse parameter to stop this behaviour. But how can we do that in ml python sdk v2.
I used the following code to create a step:
data_prep_component = command(
name="data_prep_credit_defaults",
display_name="Data preparation for training",
description="Preprocessing the training data",
inputs={
#"input_dir": Input(type="string"),
"output_dir": Input(type="string"),
},
outputs={
#"output_dir": Output(type="string"),
},
# The source folder of the component
code=PATH,
command="""python datapreprocessor.py \
--output_dir ${{inputs.output_dir}} \
""",
environment="bp_environment:3.0",
)
Issue Analytics
- State:
- Created a year ago
- Comments:8 (3 by maintainers)
Top Results From Across the Web
allow_reuse in Azure ml sdk v2 - Microsoft Q&A
I am running a training pipeline on azure ml python sdk v2. ... to finish which I assume is taking previous step from...
Read more >Tutorial: ML pipelines with Python SDK v2 - Azure
You'll learn how to use the AzureML Python SDK v2 to: Connect to your Azure ... You'll then run the pipeline, deploy the...
Read more >Create and run ML pipelines - Azure Machine Learning
Create the resources required to run an ML pipeline: Set up a datastore used to access the data needed in the pipeline steps....
Read more >Upgrade parallel run step to SDK v2 - Azure Machine Learning
Use parallel_run_function to create parallel job by replacing ParallelRunConfig and ParallelRunStep in v1. · Migrate your v1 pipeline to v2. Then ...
Read more >Upgrade pipelines to SDK v2 - Azure Machine Learning
In this article. Run a pipeline; Mapping of key functionality in SDK v1 and SDK v2; Step and job/component type mapping; Related documents....
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Thanks for the feedback. Please use is_deterministic argument.
Hi @rjtshrm, since you haven’t asked that we “
/unresolve
” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve
” to reopen the issue.