Uploading 'empty' blocks to page blobs is wasteful.
See original GitHub issueRe: PageBlobService#create_blob_from_stream
When a page blob is provisioned, the content is a stream of 0s. Therefore, when uploading a page blob, any block that is all zeros does not need to be uploaded, it can simply be skipped.
Most fixed-length VHD images are mostly empty blocks, so uploading all those zeros represents a huge waste of time and bandwidth.
Our optimum solution for dealing with this in azurectl was to load a block from /dev/zero
and compare it to the current block in the stream. If they are equal… move along to the next block without uploading.
See https://github.com/SUSE/azurectl/blob/master/azurectl/storage/page_blob.py#L50 for example.
And, just for reference, a 30GB openSUSE Leap 42.3 image, contains < 300MB of full blocks. Uploading with the API convenience methods, or az
, results in an upload of 29.7GB of zeros. o.0
Issue Analytics
- State:
- Created 6 years ago
- Reactions:1
- Comments:7 (5 by maintainers)
@bear454 FYI this feature has been implemented in commit https://github.com/Azure/azure-storage-python/commit/eb9183a20ba67cdbab868df3241258e93eb3cc3f
This was fixed in release 1.0.0 of the blob package.