New region ap-east-1 causes issues in describe_instances

We have ec2 instances in multiple regions. We have a scripts that fetches all our instances in all regions.

` ec2client = boto3.client(‘ec2’, region_name=‘eu-west-1’) response = ec2client.describe_regions()

for region in response[‘Regions’]: ec2client = boto3.client(‘ec2’, region_name=region[‘RegionName’]) response = ec2client.describe_instances() `

This fails since the new region in Hong Kong is open: https://aws.amazon.com/blogs/aws/now-open-aws-asia-pacific-hong-kong-region/

Note that we do not have the new region enabled. describe_regions includes it however. The documentation says ‘regions available to you’.

Error message:

botocore.hooks: DEBUG: Event before-call.ec2.DescribeInstances: calling handler <function inject_api_version_header_if_needed at 0x7fcf5faa1b90> botocore.endpoint: DEBUG: Making request for OperationModel(name=DescribeInstances) with params: {‘body’: {‘Action’: u’DescribeInstances’, ‘Version’: u’2016-11-15’}, ‘url’: u’https://ec2.ap-east-1.amazonaws.com/‘, ‘headers’: {‘Content-Type’: ‘application/x-www-form-urlencoded; charset=utf-8’, ‘User-Agent’: ‘Boto3/1.9.136 Python/2.7.12 Linux/4.4.0-146-generic Botocore/1.12.136’}, ‘context’: {‘auth_type’: None, ‘client_region’: ‘ap-east-1’, ‘has_streaming_input’: False, ‘client_config’: <botocore.config.Config object at 0x7fcf5a43a5d0>}, ‘query_string’: ‘’, ‘url_path’: ‘/’, ‘method’: u’POST’} botocore.hooks: DEBUG: Event request-created.ec2.DescribeInstances: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7fcf5a43a590>> botocore.hooks: DEBUG: Event choose-signer.ec2.DescribeInstances: calling handler <function set_operation_specific_signer at 0x7fcf5fa9f320> botocore.auth: DEBUG: Calculating signature using v4 auth. botocore.auth: DEBUG: CanonicalRequest: POST /

content-type:application/x-www-form-urlencoded; charset=utf-8 host:ec2.ap-east-1.amazonaws.com x-amz-date:20190426T090746Z

content-type;host;x-amz-date 6171eb09865e32b0602af0f7957e26573a51f53caaedff02ff88883cb0275885 botocore.auth: DEBUG: StringToSign: AWS4-HMAC-SHA256 20190426T090746Z 20190426/ap-east-1/ec2/aws4_request 4dba02c467b70e85d9856c14730f7e21bd85df96af3df7a495901cc429b79a8b botocore.auth: DEBUG: Signature: 41370e62e65a0d3178d4e7990a89faedc26bbbbecc392204819aace2597f15ad botocore.endpoint: DEBUG: Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://ec2.ap-east-1.amazonaws.com/, headers={‘Content-Length’: ‘43’, ‘Content-Type’: ‘application/x-www-form-urlencoded; charset=utf-8’, ‘Authorization’: ‘AWS4-HMAC-SHA256 Credential=AKIAI6HSHCH6N3HC2RIA/20190426/ap-east-1/ec2/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=41370e62e65a0d3178d4e7990a89faedc26bbbbecc392204819aace2597f15ad’, ‘X-Amz-Date’: ‘20190426T090746Z’, ‘User-Agent’: ‘Boto3/1.9.136 Python/2.7.12 Linux/4.4.0-146-generic Botocore/1.12.136’}> urllib3.util.retry: DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0, status=None) urllib3.connectionpool: DEBUG: Starting new HTTPS connection (1): ec2.ap-east-1.amazonaws.com:443 urllib3.connectionpool: DEBUG: https://ec2.ap-east-1.amazonaws.com:443 “POST / HTTP/1.1” 401 None botocore.parsers: DEBUG: Response headers: {‘Transfer-Encoding’: ‘chunked’, ‘Date’: ‘Fri, 26 Apr 2019 09:07:47 GMT’, ‘Server’: ‘AmazonEC2’} botocore.parsers: DEBUG: Response body:

<?xml version="1.0" encoding="UTF-8"?>

<Response><Errors><Error>AuthFailure<Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>9b981b0f-930f-48e7-baab-8914d0834889</RequestID></Response>

[SpenGietz]

Is there a potential fix for this when using boto3.session.Session().get_available_regions? I’m running into errors with services that don’t have a describe_regions API call.

[thejazzid]

here is AWS solution https://aws.amazon.com/premiumsupport/knowledge-center/iam-validate-access-credentials/