question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BlobStorage: Possibility of downloading a batch of blobs

See original GitHub issue

Which service(blob, file, queue) does this issue concern?

blob

Which version of the SDK was used? Please provide the output of pip freeze.

azure-storage-blob==1.4.0 azure-storage-common==1.4.0

What problem was encountered?

I am missing the possibility of downloading/reading a batch of blobs (specifically a “folder”) – a functionality which is already present in the Azure CLI (az storage blob download-batch). I am currently using the service.get_blob_to_text(container: str, path: str), and I imagine the syntax being service.get_blobs_to_text(container: str, paths: iterable).

Have you found a mitigation/solution?

Currently I am reading the files one-by-one, which generates a performance overhead.

Issue Analytics

  • State:open
  • Created 5 years ago
  • Reactions:7
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

4reactions
zezha-msftcommented, Feb 12, 2019

Hi @casparjespersen, thanks for reaching out!

We already have this feature on our roadmap, but I unfortunately do not have a timeline to share yet.

1reaction
erfannarimancommented, Dec 10, 2020

I created a Python wrapper for the Azure CLI to do downloads / uploads in batches, see here for more details:

Installation:

pip install azurebatchload

See the GitHub repo for usage examples.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Performance and scalability checklist for Blob storage
A checklist of proven practices for use with Blob storage in developing high-performance applications.
Read more >
How to Use Blob Storage via Azure File Storage
Select the file (temp.dat) to download and click on the Download button in the Azure Storage Explorer, as shown below. Doing so opens...
Read more >
Number of threads for Azure blobs - SAP Help Portal
For example, when the Number of threads is set to 16 for a large file download to your local container, the software divides...
Read more >
Bulk download from Azure Blob Storage with C# - elmah.io Blog
The GetBlobs method returns a list of blob metadata that can be sequentially downloaded using the GetBlobClient method and some streaming magic.
Read more >
azurebatchload · PyPI
Azure Batch Load ... High level Python wrapper around the Azure CLI to download or upload files in batches from or to Azure...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found