Bug: API Gateway Console Test button fails validation with @event_parser because requestContext -> identity -> sourceIp value is not a valid IPv4 or IPv6 network

Expected Behaviour

We should be able to generate a test form the API Gateway console that allows the ApiGatewayProxyEventModel to be validated and parsed by Pydantic.

Current Behaviour

When generating a test from the API Gateway console, the value for the sourceIp in requestContext -> identity is "test-invoke-source-ip". Since this value is defined to be IPvAnyNetwork here, the validation fails in the event_parser decorator here. And so test events from the API Gateway console fail with a 502 error because of the resulting pydantic ValidationError.

Code snippet

Create a lambda handler with an event parser like this…

from aws_lambda_powertools.utilities.parser import event_parser
from aws_lambda_powertools.utilities.parser.models import APIGatewayProxyEventModel
from aws_lambda_powertools.utilities.typing import LambdaContext

@logger.inject_lambda_context(log_event=settings.debug)
@event_parser(model=APIGatewayProxyEventModel)
def handler(event: APIGatewayProxyEvent, context: LambdaContext) -> dict[str, Any]:
    logger.info('event: %s', event.json())
    return {'statusCode': 200, 'body': json.dumps({'wow': 42})}

Use the following message to the lambda:

{
    "level": "INFO",
    "location": "decorate:352",
    "message": {
        "resource": "/v1/redacted/12345",
        "path": "/v1/redacted/12345",
        "httpMethod": "GET",
        "headers": {
            "Accept": "application/json"
        },
        "multiValueHeaders": {
            "Accept": [
                "application/json"
            ]
        },
        "queryStringParameters": null,
        "multiValueQueryStringParameters": null,
        "pathParameters": {
            "redacted_id": "12345"
        },
        "stageVariables": null,
        "requestContext": {
            "resourceId": "redacted",
            "resourcePath": "/v1/redacted/{redacted_id}",
            "httpMethod": "GET",
            "extendedRequestId": "Redacted=",
            "requestTime": "30/Sep/2022:19:19:03 +0000",
            "path": "/v1/redacted/{redacted_id}",
            "accountId": "012345678912",
            "protocol": "HTTP/1.1",
            "stage": "test-invoke-stage",
            "domainPrefix": "testPrefix",
            "requestTimeEpoch": 1664565543376,
            "requestId": "9c4368e8-8510-42f1-875b-ecb63f3f5c5a",
            "identity": {
                "cognitoIdentityPoolId": null,
                "cognitoIdentityId": null,
                "apiKey": "test-invoke-api-key",
                "principalOrgId": null,
                "cognitoAuthenticationType": null,
                "userArn": "arn:aws:sts::012345678912:assumed-role/redacted/joe.bagadonuts",
                "apiKeyId": "test-invoke-api-key-id",
                "userAgent": "aws-internal/3 aws-sdk-java/1.12.302 Linux/5.4.207-126.363.amzn2int.x86_64 OpenJDK_64-Bit_Server_VM/25.342-b07 java/1.8.0_342 vendor/Oracle_Corporation cfg/retry-mode/standard",
                "accountId": "012345678912",
                "caller": "REDACTED:joe.bagadonuts",
                "sourceIp": "test-invoke-source-ip",
                "accessKey": "REDACTED",
                "cognitoAuthenticationProvider": null,
                "user": "REDACTED:joe.bagadonuts"
            },
            "domainName": "testPrefix.testDomainName",
            "apiId": "1234ASFD"
        },
        "body": null,
        "isBase64Encoded": false
    },
    "timestamp": "2022-09-30 19:19:03,390+0000",
    "service": "redacted-redacto",
    "cold_start": false,
    "function_name": "redacted",
    "function_memory_size": "128",
    "function_arn": "arn:aws:lambda:us-east-1:012345678912:function:redacted",
    "function_request_id": "bcf14ad4-c46f-471c-8b9f-60e56932c9e6",
    "xray_trace_id": "9-12345678-1234asdf2345qwer"
}

Note the error that the sourceIp is not an IPv4 or IPv6 value.

Possible Solution

Update the sourceIp here from this:

    sourceIp: IPvAnyNetwork

to this:

    sourceIp: Union[IPvAnyNetwork, Literal['test-invoke-source-ip']]

You may want to add a test (maybe in this file?) that tests for the 'test-invoke-source-ip' value explicitly.

Note this may also be an issue with the APIGatewayProxyEventV2Model because of the sourceIp definition here.

Steps to Reproduce

Create the lambda function provided above. Deploy the lambda function provided above behind an api gateway. Use the api gateway console to test the function passing in the redacted_id of 12345, and Accept:application/json for the headers.

AWS Lambda Powertools for Python version

latest

AWS Lambda function runtime

3.9

Packaging format used

PyPi

Debugging logs

No response

[heitorlessa]

Update. I have good news to share @mikelane. API Gateway Engineering team will fix the source by the end of the month - no more test-invoke-source-ip value.

We’ll update here when it’s done and confirmed.

[heitorlessa]

Thank you @mikelane for flagging this issue in the API Gateway service.

Would you be able to create a support case to suggest API Gateway team to use an actual IP Address in the test event?

We’ll cut an internal ticket to them next week.

I’m removing the bug label as Parser is behaving as expected. That said, we should fix this at the source, otherwise this will happen again next time this changes to another literal we have no control over.

If helpful, we keep track of real events for each event source we support here: https://github.com/awslabs/aws-lambda-powertools-python/blob/develop/tests/events/

Have a great weekend!