(lambda-python): arm64 architecture is not respected

What is the problem?

Hey,

Not quite sure if that’s a problem with lambda or with codepipelines or at all with cdk. Been trying to understand it but I just couldn’t. Here’s what is happening.

I have a pipelines.CodePipeline that is configured with code_build_defaults that states that I wish to run builds with LinuxBuildImage.AMAZON_LINUX_2_ARM_2. Obviously later on my lambda functions are configured as follow:

PythonFunction(..., runtime=Runtime.PYTHON_3_9, architecture=Architecture.ARM_64)

That being said I expect that when CodeBuilds runs my synth step it will run on top of arm64 and lambda will be build using the same architecture. However in the logs of job I see this:

111 | Status: Downloaded newer image for public.ecr.aws/sam/build-python3.9:latest
112 | ---> 7925e2bf1015
113 | Step 3/8 : ARG PIP_INDEX_URL
114 | ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
115 | ---> Running in 47078dfc29f5
116 | Removing intermediate container 47078dfc29f5
117 | ---> b95a315c94b2
118 | Step 4/8 : ARG PIP_EXTRA_INDEX_URL
119 | ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
120 | ---> Running in 684bb68c4f18
121 | Removing intermediate container 684bb68c4f18
122 | ---> 175da3dd341e
123 | Step 5/8 : ARG HTTPS_PROXY
124 | ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
125 | ---> Running in 15976d15e0ae
126 | Removing intermediate container 15976d15e0ae
127 | ---> 28715ae8a9d6
128 | Step 6/8 : RUN pip install --upgrade pip
129 | ---> [Warning] The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
130 | ---> Running in 515be8ae3321
131 | standard_init_linux.go:228: exec user process caused: exec format error
132 | The command '/bin/sh -c pip install --upgrade pip' returned a non-zero code: 1
133 | jsii.errors.JavaScriptError:

Which seems to contradict to my desire setup of arm64. I expected public.ecr.aws/sam/build-python3.9:latest-arm64 would be downloaded instead of public.ecr.aws/sam/build-python3.9:latest (lack or arm64) in the end.

That happens with cdk 2.8.0. Last successful build I managed to execute with here was done with cdk 1.134.0.

PS. My stack defines also AwsCustomResoure which is another lambda deployment. Might it be that arm64 and custom resource lambda conflicts each other?

Reproduction Steps

import typing as t

import aws_cdk as cdk
from aws_cdk import (
    aws_codebuild as cb,
    aws_codepipeline as cp,
    aws_iam as iam,
    aws_lambda as fn,
    aws_lambda_python as fn_python,
    pipelines,
)
from constructs import Construct

class Pipeline(cdk.Stack):
    def __init__(
        self,
        scope: Construct,
        construct_id: str,
        **kwargs: t.Any,
    ) -> None:
        super().__init__(
            scope,
            construct_id,
            **kwargs,
        )

        connection_arn =  '' # some connection arn here
        repo_name = '' # some repo name here

        pipeline = pipelines.CodePipeline(
            self,
            'Pipeline',
            pipeline_name='IAMSetupPipeline',
            synth=pipelines.ShellStep(
                'Synth',
                input=pipelines.CodePipelineSource.connection(
                    repo_name
                    'master',
                    connection_arn=connection_arn,
                ),
                install_commands=[
                    'npm install -g aws-cdk@latest',
                    'pip install -r requirements.txt',
                ],
                commands=['cdk synth pipeline'],
            ),
            code_build_defaults=pipelines.CodeBuildOptions(
                build_environment=cb.BuildEnvironment(
                    build_image=cb.LinuxBuildImage.AMAZON_LINUX_2_ARM_2,
                ),
            ),
            self_mutation=True,
            docker_enabled_for_synth=True,
        )

        # stages
        pipeline.add_stage(
            AppStage(
                self,
               'bug',
                env=cdk.Environment(
                    account='000000000000',
                    region='eu-central-1',
                ),
            ),
        )

        pipeline.build_pipeline()


class AppStage(cdk.Stage):
    def __init__(
        self,
        scope: Construct,
        stage_id: str,
        **kwargs: t.Any,
    ) -> None:
        super().__init__(scope, stage_id, **kwargs)
        LambdaStack(self, 'stack')

class LambdaStack(cdk.Stack):
    def __init__(
        self,
        scope: Construct,
        construct_id: str,
        **kwargs: t.Any,
    ) -> None:
        super().__init__(
            scope,
            construct_id,
            **kwargs,
        )
 
        fn_python.PythonFunction(
            self,
            'ARM',
            entry='/your/entry',
            runtime=fn.Runtime.PYTHON_3_9,
            architecture=fn.Architecture.ARM_64,
        )

app = cdk.App()
Pipeline(
    app, 
    'Pipeline', 
    env=cdk.Environment(
                    account='000000000000',
                    region='eu-central-1',
))
app.synth()

CODE is not exactly to be run as-is. It lacks a python lambda code (this is what I have been using) and it lacks proper account and region

What did you expect to happen?

Stack is built correctly inside of deployed pipeline and ARM64 is used.

What actually happened?

Stack fails to built with code linked in first section.

CDK CLI Version

2.8.0

Framework Version

No response

Node.js Version

14.17.5

OS

MacOs BigSur

Language

Python

Language Version

3.10.1

Other information

If I messed something up. I am sorry 😦

[kornicameister]

@ryparker shouldn’t this actually be p1. Not familiar with assessment process but based on other issues I do feel like cdk-team prioritizes issues when something that used to work stopped with p1 which means higher prio and someone from core team looking at this?

[hxy1991]

fn := awslambdapython.NewPythonFunction(stack, jsii.String("ddb-to-opensearch"), &awslambdapython.PythonFunctionProps{
	Architecture: awslambda.Architecture_ARM_64(),
	Runtime:      awslambda.NewRuntime(jsii.String("python3.9:latest-arm64"), awslambda.RuntimeFamily_PYTHON, nil),
	Entry:        jsii.String(codeDir),
	LogRetention: awslogs.RetentionDays_ONE_MONTH,
})

We solved this problem by setting “Bundling”.

fn := awslambdapython.NewPythonFunction(stack, jsii.String("ddb-to-opensearch"), &awslambdapython.PythonFunctionProps{
	Architecture: awslambda.Architecture_ARM_64(),
	Runtime: awslambda.Runtime_PYTHON_3_9(),
	Entry:        jsii.String(codeDir),
	LogRetention: awslogs.RetentionDays_ONE_MONTH,

	// We solved this problem by setting "Bundling"
	Bundling: &awslambdapython.BundlingOptions{
		Image: awscdk.DockerImage_FromRegistry(jsii.String("public.ecr.aws/sam/build-python3.9:latest-arm64")),
	},

	……
})