Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
AzureML OutputFileDatasetConfig not useable with destination set
See original GitHub issue- Package Name: azureml-core
- Package Version: 1.21.0.post1
- Operating System: MacOS
- Python Version: 3.7.4
Describe the bug
I am not able to use OutputFileDatasetConfig with a destination pointing to a Blobstorage or DataLake Gen 2.
The following error is raised:
File "/.venv/lib/python3.7/site-packages/azureml/data/output_dataset_config.py", line 459, in to_data_location
datastore_name = destination[0].name
TypeError: 'AzureBlobDatastore' object is not subscriptable
To Reproduce Steps to reproduce the behavior:
# Connect to Azure ML
interactive_auth = InteractiveLoginAuthentication(tenant_id=tenant_id)
ws = Workspace(subscription_id, resource_group, workspace_name, auth=interactive_auth)
experiment = Experiment(workspace=ws, name="test-experiment")
# Get Dataset
my-output-dataset = OutputFileDatasetConfig(name="attachments", destination=ws.datastores["my-datastore"])
# Set Run Config
pipeline_path = Path(__file__).parent.absolute()
config = ScriptRunConfig(
source_directory='./',
script="my-script.py",
compute_target="default-compute-target",
arguments=[
"--path",
my-output-dataset.as_mount(),
],
)
# Execute Script
run = experiment.submit(config)
Expected behavior Mount Blobstorage and Datalake Gen 2 as output dataset to a compute target
Screenshots
Additional context
Issue Analytics
- State:
- Created 3 years ago
- Comments:11 (1 by maintainers)
Top Results From Across the Web
Moving data in ML pipelines - Azure Machine Learning
This policy can only be set to a workspace's non-default datastore. Use OutputFileDatasetConfig to export intermediate data to another datastore ...
Read more >'AzureBlobDatastore' object is not subscriptable - Stack Overflow
I'm trying to create an azure ml pipeline with python steps and I ... AzureML OutputFileDatasetConfig not useable with destination set.
Read more >Data | Azure Machine Learning
Guide to working with data in Azure ML. ... Read from Datastore#. Reference data in a Datastore in your code, for example to...
Read more >ML Pipelines in Azure Machine Learning the right way - Medium
Adding all AML specific code to an azureml folder avoids polluting your non-cloud-vendor code. This renders the general and the azure-specific ...
Read more >The missing guide to AzureML, Part 3: Connecting to data and ...
Welcome to our series on using Azure Machine Learning. ... The missing guide to AzureML, Part 1: Setting up your AzureML workspace ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Thanks for the clarification!
Might be a good idea to improve the error messages.
Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.
Issue Details
Describe the bug I am not able to use
OutputFileDatasetConfigwith a destination pointing to a Blobstorage or DataLake Gen 2.The following error is raised:
To Reproduce Steps to reproduce the behavior:
Expected behavior Mount Blobstorage and Datalake Gen 2 as output dataset to a compute target
Screenshots
Additional context
Client,Machine Learning,Service Attention,bug,customer-reported,needs-triage,question