Presigned url not creating a valid URL for `af-south-1`

Describe the bug Presigned url does not create a url for the region_name specified

Steps to reproduce

Its worth noting the code I am working on has its only mechanism for storing secrets, so they are retrieved as the variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_STORAGE_BUCKET_NAME.

1. Create a bucket in the region af-south-1
2. Place a file in it.
3. Try generate a presigned URL with the following code:
4. I only specify the region_name again when creating the resouce because its not working:

import boto3

session = boto3.Session(
        aws_access_key_id=AWS_ACCESS_KEY_ID,
        aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
        region_name='af-south-1')
print(session.region_name)  # prints af-south-1
resource = session.resource('s3', region_name='af-south-1')
response = resource.meta.client.generate_presigned_url(
        'get_object',
        Params={'Bucket': AWS_STORAGE_BUCKET_NAME,  'Key': TEST_FILE_NAME},
        ExpiresIn=3600,
    )
print(response)

1. It will generate a url like:
2. https://bucket-name.s3.amazonaws.com/test1.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA32EU3DKFB3ACLEUV%2F20210928%2Faf-south-1%2Fs3%2Faws4_request&X-Amz-Date=20210928T130419Z&X-Amz-SignedHeaders=host&X-Amz-Expires=1800&X-Amz-Signature=0b684c8f423c9846cac3d437e3444972e2ead4f58ea967bc50c4942534aacf01

Which if tried gives an error:

<Error>
    <Message>
    <Code>IllegalLocationConstraintException</Code>
        The af-south-1 location constraint is incompatible for the region specific endpoint this request was sent to.
    </Message>
    <RequestId>PJJQ0T7PAQHWQK5S</RequestId>
    <HostId>
        LmcUnFY3JJSd03OMzsGmxYO/tvl/PpYrEO6TtVI80AJiWHPrpge20gllm+RUkKC3ejuGwy8ZpG0=
    </HostId>
</Error>

Expected behavior A url more like af-south-1.amazonaws.com

Related to #2098, however I set the region name as shown above.

[eNcacz]

Possible workaround without hardcoding the URL structure:

s3 = boto3.client('s3', region_name='af-south-1')
endpointUrl = s3.meta.endpoint_url
s3 = boto3.client('s3', endpoint_url=endpointUrl, region_name='af-south-1')

I know, it is not nice, the fix in boto3 would be much better, but it works and no hardcoding is needed.

[jonemo]

Another workaround is to use an S3 Access Point in place of the bucket name.

For example, let’s assume you have a bucket named bucketname in af-south-1 and create an access point named bucketname-ap for the bucket. You can then find the access point’s ARN from the AWS Console or using the s3-control list_access_points API and use it in place of the bucket name:

s3 = boto3.client('s3', region_name='af-south-1')
access_point_arn = 'arn:aws:s3:af-south-1:01234567890:accesspoint/bucketname-ap'
url = s3.generate_presigned_url('get_object', Params={'Bucket': access_point_arn, 'Key': 'object.txt'})

This will produce a URL in the correct regional format:

https://[prefix].s3-accesspoint.af-south-1.amazonaws.com/object.txt?[parameters]

In fact, you don’t have to specify the region_name during client creation because boto3 will use the region name from the access point ARN.

That said, I agree that a fix in boto3 would be preferable to any workarounds. Further investigation is needed to understand exactly when the current behavior does not work to ensure that any change we make does not also modify currently working inputs.