Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
list_blob_names is slow on my storage, returns markers
See original GitHub issueWhich service(blob, file, queue) does this issue concern?
blob
Which version of the SDK was used? Please provide the output of pip freeze.
azure-storage-blob==2.1.0 azure-storage-common==2.1.0 azure-storage-file==2.1.0
What problem was encountered?
list_blob_names is extremely slow for a particular blob as it seems to be listing “markers” , never seen that before, I just call list_blob_names without num_results.
The response object is an empty list and a marker:
x-ms-date:Thu, 28 May 2020 15:43:56 GMT x-ms-version:2019-02-02 /alsordon/phillytools comp:list delimiter:/ marker:2!472!MDAwMzA5IXByb2plY3RzL2luZm9uY2VzL2MxMF9yZXMxOC9wdC1yZXN1bHRzL2FwcGxpY2F0aW9uXzE1ODMzMDcxNTM4NjhfMzIzNjMvU3VwQ29uL2NpZmFyMTBfdGVuc29yYm9hcmQvSW5mb05DRXNfY2lmYXIxMF9yZXNuZXQxOF9scl8wLjVfZGVjYXlfMC4wMDAxX2Jzel8zMDcyX3RlbXBfMC41X2wxMl8xLjAwX2wxY3V0Ml8xLjAwX2wxMkkxY3V0XzEuMDBfbnNfNTEyX3Zhcl8wLjEwMF90cmlhbF8wX2Nvc2luZV93YXJtL2V2ZW50cy5vdXQudGZldmVudHMuMTU5MDM0ODMxOS5jb250YWluZXItZTc3OC0xNTgzMzA3MTUzODY4LTMyMzYzLTAxLTAwMDAwMiEwMDAwMjghMjAyMC0wNS0yNVQwMzo0Mzo0OS41NjEyMDg5WiE- prefix:projects/infonces/ restype:container 2020-05-28 11:44:01 AM - DD https://alsordon.blob.core.windows.net:443 “GET /phillytools?restype=container&comp=list&prefix=projects%2Finfonces%2F&delimiter=%2F&marker=2%21472%21MDAwMzA5IXByb2plY3RzL2luZm9uY2VzL2MxMF9yZXMxOC9wdC1yZXN1bHRzL2FwcGxpY2F0aW9uXzE1ODMzMDcxNTM4NjhfMzIzNjMvU3VwQ29uL2NpZmFyMTBfdGVuc29yYm9hcmQvSW5mb05DRXNfY2lmYXIxMF9yZXNuZXQxOF9scl8wLjVfZGVjYXlfMC4wMDAxX2Jzel8zMDcyX3RlbXBfMC41X2wxMl8xLjAwX2wxY3V0Ml8xLjAwX2wxMkkxY3V0XzEuMDBfbnNfNTEyX3Zhcl8wLjEwMF90cmlhbF8wX2Nvc2luZV93YXJtL2V2ZW50cy5vdXQudGZldmVudHMuMTU5MDM0ODMxOS5jb250YWluZXItZTc3OC0xNTgzMzA3MTUzODY4LTMyMzYzLTAxLTAwMDAwMiEwMDAwMjghMjAyMC0wNS0yNVQwMzo0Mzo0OS41NjEyMDg5WiE- HTTP/1.1” 200 None 2020-05-28 11:44:01 AM - DD String_to_sign=GET
x-ms-client-request-id:0da62ca2-a0fa-11ea-a479-554857c9fd70 x-ms-date:Thu, 28 May 2020 15:44:01 GMT x-ms-version:2019-02-02 /alsordon/phillytools comp:list delimiter:/ marker:2!400!MDAwMjU1IXByb2plY3RzL2luZm9uY2VzL2MxMF9yZXMxOC9wdC1yZXN1bHRzL2FwcGxpY2F0aW9uXzE1ODMzMDcxNTM4NjhfMzIzNjYvU3VwQ29uL2NpZmFyMTBfdGVuc29yYm9hcmQvU2ltQ0xSX2NpZmFyMTBfcmVzbmV0MThfbHJfMC41X2RlY2F5XzAuMDAwMV9ic3pfMzA3Ml90ZW1wXzAuNV90cmlhbF8wX2Nvc2luZV93YXJtL2V2ZW50cy5vdXQudGZldmVudHMuMTU5MDM0ODQyMC5jb250YWluZXItZTc3OC0xNTgzMzA3MTUzODY4LTMyMzY2LTAxLTAwMDAwMiEwMDAwMjghMjAyMC0wNS0yNFQyMjowNjo1NS40NzUzMDk4WiE- prefix:projects/infonces/ restype:container
Have you found a mitigation/solution?
No
Note: for table service, please post the issue here instead: https://github.com/Azure/azure-cosmosdb-python.
Issue Analytics
- State:
- Created 3 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
Hi @sordonia
Thanks for reaching out! Are you doing something like this next(self.bs.list_blobs(container_name))? Just want to clarify that the “log” you pasted there are requests. If there’s a marker in request that means the response returned that marker, then the response should have returned the list of blobs.
BTW you said “never seen that before” so does that mean this was working and suddenly it’s not working? if that’s the case we will look into service log and see what happens!
Thanks!
Hi @erlisb azure-storage has been deprecated, would you like to use azure-storage-blob<=2.1.0 or azure-storage-blob>=12.0.0 (there are be breaking changes if you use >=12.0.0)
@zezha-msft do you have any idea about the list_blobs behavior in azure-storage==0.36…