Custom Headers in Presigned URLs Construction

Python: 2.7.5 Boto: 2.48.0 Boto3: 1.4.7 Botocore: 1.7.19

Query String Request Authentication Alternative specifies the format of the StringToSign that might be followed to create a signature of the pre-signed url. Namely,

Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) ) );

StringToSign = HTTP-VERB + "\n" +
    Content-MD5 + "\n" +
    Content-Type + "\n" +
    Expires + "\n" +
    CanonicalizedAmzHeaders +
    CanonicalizedResource;    

The use case is to be able to provide a custom header like Content-MD5 to participate in signature construction.

This is perfectly fine with the boto library

import boto 

## [s3]
## use-sigv4 = True
boto.config.load_from_path('.boto')

conn = boto.connect_s3(profile_name = "_profile_name_", host = "s3.amazonaws.com")

presigned_url = conn.generate_url(
	expires_in=3600,
	method='GET',
	bucket='_bucket_name_',
	key='_key_',
	headers={'Content-MD5': 'c3993a7637ae5d90e70106050ef6926f'}
)

Seems like this same feature cannot be achieved with boto3.

Can this be confirmed and what is the reason behind degrading this feature from the new version of the library? May it be expected to return in the future versions?

[kasured]

I would rather we do keep separate flavors of the issue described here. I specifically stressed out the possibility to have the signature be customized in terms of adding additional components that participate in the process.

Content-MD5 being described as “meaningless” in the GET requests does not disallow it to be present. After all, empty value still gives you a hash albeit deterministic.

The use case described just brings up some additional custom contract to the process. If say Alice and Bob decide to include the custom header in the process of signing url requests, than Charlie being not a part of that contract will have an additional step to be able to get a pre-signed url working for him.

There are lots of scenarios that might be exampled here, without stressing out a specific header.

In the meantime, other SDKs like AWS Java SDK and a “hand-coded” boto library does support that feature without violating any strict contract from the AWS Signing Interface.

[yanivpaz]

see also https://github.com/boto/boto3/discussions/3342