[aws-lambda-python]: Ability to customize build environment

Allow the ability to customize the environment Python builds are done within, whilst still taking advantage of the simplicity of what PythonFunction provides.

Specifically, without requiring us to provide a custom docker image, allow us to specify custom docker volumes, and custom shell commands to run before the build.

This is a similar but different requirement to Allow the use of CodeArtifact, and it may be that the same solution applies for both use cases.

Please note that I’m explicitly avoiding customisation of the Docker image build; this is customisation of how the python build is executed using a Docker image.

Use Case

I want my Lambda code to be able to have dependencies on packages in Github private repositories. To allow from this, I want to be able to copy my SSH keys from my host machine into the build Docker volume, so that the build can authenticate to Github using my SSH keys.

Proposed Solution

Allow for syntax something like:

    aws_lambda_python.PythonFunction(
        scope=self,
        id="FunctionId",
        handler="handler",
        runtime=aws_lambda.Runtime.PYTHON_3_8,
        entry="source-entry",
        prebuild_command=[
            "bash",
            "-c",
            "cp -r /tmp/ssh/* ~/.ssh/",
        ],
        build_docker_volumes=[
            core.DockerVolume(
                container_path="/tmp/ssh",
                host_path=f"{Path.home()}/.ssh",
            ),
        ],
    )

---

This is a 🚀 Feature Request

[setu4993]

which I believe would require the ability to at least run something similar the following on the container,

ADD dir-containing-intercepting-cert /usr/local/share/ca-certificates
RUN update-ca-certificates

add the additional lines needed to copy in my SSH keys

FWIW, @SamStephens and @DarrenForsythe, Docker recently started supporting passing in SSH keys using build secrets, that might be a more secure way than copying them in / using volumes.

[setu4993]

I guess my one minor quibble is that if I understand you correctly, to use your feature to (for example) customise to include my SSH keys, I’m going to have to duplicate the contents of Dockerfile.dependencies in my custom Dockerfile.dependencies and then add the additional lines needed to copy in my SSH keys. To me this isn’t ideal, because it leaves room for your customised Dockerfile.dependencies to drift apart from the canonical aws-lambda-python Dockerfile.dependencies as the CDK evolves. I’d prefer to be able to specify additional commands to include in the canonical aws-lambda-python Dockerfile.dependencies.

I absolutely agree. My ideal case would be to have that be easier to customize and reuse instead of copied over and then customized.

One of the reasons I think the current setup of Dockerfile.dependencies and Dockerfile is better than the one for aws_lambda.Function is because it splits the step into 2 parts: One for getting dependencies, another for getting function code. Splitting over to a different setup just for dependencies is nice because that can be customized depending on project-level requirements rather than CDK-level.

That allows for the ability to customize a few different things:

1. Customizing how packages get installed (pip, poetry, something else).
2. Adding credentials to the build step (Code Arifact, SSH keys, etc.).
3. Adding caching.
4. Passing in custom build secrets, environment variables.

We usually run in a build environment that doesn’t require pipenv (which PythonFunction supports). So, for our use case, we’d likely just drop that line (and conditionals) from our custom Dockerfile.dependencies.

It’s not a perfect solution and still relies on some duplication, yes, but it affords enough flexibility that IMHO makes it worth it.

Again, all of this is contingent on the PR being accepted 😃.