[Azure ML SDK v2] Method to download Data asset locally

The Data class in the Azure ML SDK v2 allows the uploading and creation of a new Data asset, but not its downloading. I understand that the idea is to not use the new SDK inside training jobs. However, for exploration purposes it is very handy to be able to download a registered Data asset, as is possible with the SDK v1.

Would it be possible to add this feature? Alternatively, is there a way (using other parts of the SDK?) to download assets with paths such as azureml://datastores/<data_store_name>/paths/<path>?

---

Document details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 112731d7-568e-4310-ad10-687f70e47a86
• Version Independent ID: 5ee94e1d-c95d-6def-dfbb-f31979170ed5
• Content: azure.ai.ml.entities.Data class
• Content Source: preview/docs-ref-autogen/azure-ai-ml/azure.ai.ml.entities.Data.yml
• GitHub Login: @VSC-Service-Account

[tomasvanpottelbergh]

Hi @SturgeonMi, I briefly tried this out, but couldn’t get the authentication to work. Anyway, I found a workaround using azure.ai.ml._artifacts._artifact_utilities.download_artifact_from_aml_uri. This is definitely not a great solution, since it’s a “private” API, but I hope that this functionality will get exposed publicly in the azure-ai-ml package at some point.

[jomalsan]

+1 on exposing this functionality would be great. To expand on @tomasvanpottelbergh 's solution, I was able to download locally using the following:

import os

from azure.ai.ml import MLClient
from azure.identity import DefaultAzureCredential
import azure.ai.ml._artifacts._artifact_utilities as artifact_utils

subscription_id = ""
resource_group = ""
workspace = ""

dataset_name = ""
dataset_version = ""
downloaded_data_folder = "./data"

# Get the client
ml_client = MLClient(
    DefaultAzureCredential(), subscription_id, resource_group, workspace
)

# Lookup the dataset to get the 'path'
data_info = ml_client.data.get(name=dataset_name, version=dataset_version)

# Download the dataset
artifact_utils.download_artifact_from_aml_uri(uri = data_info.path, destination = downloaded_data_folder, datastore_operation=ml_client.datastores)

# Verify it is downloaded
file_path = os.path.basename(data_info.path[10:])
assert os.path.exists(os.path.join(downloaded_data_folder, file_path))