Specifying S3 Virtual Syle Hosted Endpoint for Fips Reverts to Path Style

Describe the bug Creating an S3 client and specifying a fips endpoint URL with the virtual hosted style addressing, the client ends up reverting to path style addressing.

We understand that S3 fips endpoints are only supported with virtual hosted style addresses and that only virtual hosted style addresses will be supported for buckets created after September 30, 2020. The clients reverting to path-style addressing then becomes a problem for us.

How do we force our clients to use the virtual hosted style tips endpoints?

Steps to reproduce Client instantiation example:

s3 = boto3.client("s3", endpoint_url="https://<bucketname>.s3-fips.<region>.amazonaws.com")

Calls and testing after instantiations use path style endpoints rather than virtual hosted style.

Expected behavior The client should continue using the endpoint URL https://<bucketname>.s3-fips.<region>.amazonaws.com rather than reverting to the path style address.

Debug logs n/a

@shepdelacreme

[swetashre]

Hi @swetashre, using just the region doesn’t seem to work since we are trying to use the FIPS endpoints in the US commercial regions. We would like to just pass in the us-east-1-fips region but unfortunately, it seems like https://github.com/boto/botocore/blob/955725c257a2c9ebcb44fba383dd2860df4392ed/botocore/data/endpoints.json needs to be updated to include those regions in the s3 service block: https://github.com/boto/botocore/blob/955725c257a2c9ebcb44fba383dd2860df4392ed/botocore/data/endpoints.json#L4353

We will open a ticket in the botocore repo to get the endpoints.json file updated.

Thank you for pointing out the addressing_style config option though, it seems like we can force the construction of the FIPS url by doing this:

s3 = boto3.client('s3', endpoint_url="https://s3-fips.us-east-1.amazonaws.com", config=Config(s3={'addressing_style': 'virtual'}))

@shepdelacreme - Currently this is the only region available with s3 fips according to the endpoints.json file https://github.com/boto/botocore/blob/955725c257a2c9ebcb44fba383dd2860df4392ed/botocore/data/endpoints.json#L4353 I will create a ticket to get the endpoints file updated as this file comes from upstream.

[ddl-denis-parnovskiy]

For the customers using s3 on-prem or S3-compatible solutions like IBM COS, virtual host addressing style is not an option. Therefore forcing them to use virtual host addressing will just not work.