BlockBlobService inconsistent handling of container SAS URLs and tokens, particularly those generated by make_container_url
See original GitHub issueWhich service(blob, file, queue) does this issue concern?
Blob
Which version of the SDK was used? Please provide the output of pip freeze
.
azure-storage-blob==1.4.0
What problem was encountered?
There seems to be an inconsistency in how SAS’s are generated and managed in the Python SDK.
First let me give a bit of background about my use case. Basically, we have folks providing us a SAS to one of their containers, which we then will upload some files to. They provide the SAS to us as one big URL, that is to say something which looks like this:
https://<account>.blob.core.windows.net/<container>?se=2019-01-16T22%3A26%3A51Z&sp=w&sv=2018-03-28&sr=c&sig=<sig>
If I generate a SAS in Python like so I get a value which looks just like the query string in my sample above:
sas = blob_service.generate_container_shared_access_signature(
container_name=container_name,
permission=azure.storage.blob.ContainerPermissions(write=True),
expiry=(datetime.datetime.utcnow() + datetime.timedelta(hours=3)))
But since the URL provided to me needs to also include the container name, etc, I use blob_service.make_container_url(container_name, sas_token=sas)
to create a URL. This URL looks a bit different than I expect:
https://<account>.blob.core.windows.net/<container>?restype=container&se=2019-01-16T22%3A26%3A51Z&sp=w&sv=2018-03-28&sr=c&sig=<sig>
- specifically it’s got this extra restype=container
in it.
Later, when I want to actually perform a file upload using the provided URL, I have a few options.
- I can construct a
BlockBlobService
like this (using the full URL):blob_service = azure.storage.blob.BlockBlobService(account, sas_token='https://<account>.blob.core.windows.net/<container>?restype=container&se=2019-01-16T22%3A26%3A51Z&sp=w&sv=2018-03-28&sr=c&sig=<sig>')
- or like this (using just the query string):
blob_service = azure.storage.blob.BlockBlobService(account, sas_token='restype=container&se=2019-01-16T22%3A26%3A51Z&sp=w&sv=2018-03-28&sr=c&sig=<sig>')
Number 2 above doesn’t work, because of the restype=container
. Number 1 works, but only if the sas URL contains restype=container
- if somebody provides a URL generated from say the C# SDK which doesn’t have restype=container
I end up getting the following error when trying to use the SAS:
azure.common.AzureHttpError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. ErrorCode: AuthenticationFailed
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:72c1719b-201e-0054-078a-ae25a1000000
Time:2019-01-17T17:31:25.7442116Z</Message><AuthenticationErrorDetail>Access without signed identifier cannot have time window more than 1 hour: Start [Thu, 17 Jan 2019 17:31:25 GMT] - Expiry [Thu, 17 Jan 2019 19:29:18 GMT]</AuthenticationErrorDetail></Error>
Reading the documentation from the Python SDK, it seems like the sas_token
parameter of BlockBlobService
is supposed to just be the SAS token (i.e. this bit: se=2019-01-16T22%3A26%3A51Z&sp=w&sv=2018-03-28&sr=c&sig=<sig>
- but the problem is that when presented with a full URL containing restype=container
in the query string, it’s not clear to me how to pull out “just the SAS” given an arbitrary query string. What I had been doing was to use the full query string as the SAS, but since restype=container
is valid in the query string, but also not valid in the SAS token I’m now at a loss as to how to properly extract the SAS.
To summarize, my questions/issues are:
- Why does
blob_service.make_container_url
includerestype=container
in the URL? The URL generated already clearly points to a container so it seems like this extrarestype=container
is not useful. - Given a URL like
https://<account>.blob.core.windows.net/<container>?restype=container&se=2019-01-16T22%3A26%3A51Z&sp=w&sv=2018-03-28&sr=c&sig=<sig>
how can I extract just the sas part, so that I can then pass that to theBlockBlobService
constructor as thesas_token
parameter? - Why does
BlockBlobService
accept a full URL as thesas_token
, but it only works if that full URL containsrestype=container
? - Is the intent of the
sas_token
inBlockBlobService
that it accepts both a full URL or the bare SAS token, or should it accept only the token?
Have you found a mitigation/solution?
The only mitigation/solution I’ve found is to tell people to avoid using make_container_url
entirely, and to generate their URLs manually - which isn’t really a great solution.
Issue Analytics
- State:
- Created 5 years ago
- Reactions:2
- Comments:6 (3 by maintainers)
Top GitHub Comments
@jdthorpe - totally agree that there is a workaround… but it doesn’t make any sense to provide a helper method (
make_container_url
) which actually doesn’t help!@zezha-msft is there any plan to explain this behavior…? This issue has been open for quite a long time without a response.
Agree this is crazy to list the resource type when its already implied. Glad I found this question!