Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Partial download reported as success in SDK API get_blob_to_path()

See original GitHub issue

Which service(blob, file, queue) does this issue concern?

We are downloading the Azure blob using Azure python SDK with API get_blob_to_path(). API reports success, however data downloaded is partial.

What problem was encountered?

Below is the code snippet Note: This is the custom code snipet. The main code is the call to the SDK API get_blob_to_path() in below sample.

def download_file(self, f_context):
      src_file_path = f_context.src_path
      dest_file_path = f_context.dest_path
      az_uri = AzureUri(src_file_path)
      kek = self._blob_service.key_encryption_key
      require_encryption = self._blob_service.require_encryption
      self._blob_service.key_encryption_key = None
      self._blob_service.require_encryption = False
          blob = self._blob_service.get_blob_to_path(container_name=az_uri.container(),
          props =
          self.logger.debug('download_file content_length:%s. content_range:%s', props.content_length, props.content_range)
          self.logger.debug('download_file file size:%d', os.path.getsize(dest_file_path))
      except Exception as err:
          self.logger.error('download_file failure. Error:%s \\n StackTrace:{%s}', str(err), traceback.format_exc())
          self._blob_service.key_encryption_key = kek
          self._blob_service.require_encryption = require_encryption

Below are logs from above code path

10294: 2018-01-25 11:04:15,503 - DEBUG - lib.ms_azure.azure_file_handler:download_file - download_file content_length:171490. content_range:bytes 0-171489/171490
10294: 2018-01-25 11:04:15,504 - DEBUG - lib.ms_azure.azure_file_handler:download_file - download_file file size:48588

Below are analytic logs with removed account specific details: 1.0;2018-01-25T05:33:36.3654235Z;GetBlob;NetworkError;206;38480;20;authenticated;xxxxxx....

Question: Actual blob content length is :171490 As indicated by the analytic logs, request returned 206 (partial content) However python SDK API get_blob_to_path() reported success

Only partial data is getting downloaded while SDK returns success. This results in data corruption at our side.

Expectations: When using SDK API get_blob_to_path(), to request complete blob:

SDK API returns complete blob i.e. all the data of the blob is returned. SDK API fails i.e. it throws exception regardless of what led to underlying behaviour (network error or any other transient issue).

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:8 (5 by maintainers)

github_iconTop GitHub Comments

segulercommented, Mar 9, 2018

During a download, if a network error happens, underlying libcurl library will throw and be propagated to the Python SDK to the client.

In your test however, it seems the TCP stack is succeeding since you are manipulating the body with fiddler successfully hence there is no error thrown and propagated to the Client. This is the issue here. The test does not really simulate a real life scenario failure.

reeshabhraj1commented, May 17, 2018

Not able to repro. Closing the issue.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Partial download reported as success in SDK API ... - GitHub
We are downloading the Azure blob using Azure python SDK with API get_blob_to_path(). API reports success, however data downloaded is partial.
Read more >
Get Blob (REST API) - Azure Storage - Microsoft Learn
Indicates that the service supports requests for partial blob content. Included for requests that are made by using version 2011-08-18 and later ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found