Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Cache is invalid when using containers environment variables

See original GitHub issue

Description:

Cache is always invalid when building with containers environment variables (-e, --container-env-var)

Steps to reproduce:

Create a any sample app with sam init
Build multiple times with sam build -cu -e FOO=BAR

Observed result:

“Cache is invalid” message
Function’s hash is changing

❯ sam build -cu -e FOO=BAR
Starting Build use cache
Starting Build inside a container
Cache is invalid, running build and copying resources to function build definition of b89956ad-7a89-401a-98b5-68dad76c9c2d
Building codeuri: /private/var/folders/bf/f_z6y6n959193x3csh_pcb0m0000gn/T/tmp.Zjp9bgzU/sam-app/hello_world runtime: python3.8 metadata: {} functions: ['HelloWorldFunction']

Fetching public.ecr.aws/sam/build-python3.8:latest Docker container image......
Mounting /private/var/folders/bf/f_z6y6n959193x3csh_pcb0m0000gn/T/tmp.Zjp9bgzU/sam-app/hello_world as /tmp/samcli/source:ro,delegated inside runtime container

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
❯ sam build -cu -e FOO=BAR
Starting Build use cache
Starting Build inside a container
Cache is invalid, running build and copying resources to function build definition of fb423804-b45f-431a-9eca-e5d58909cbe6
Building codeuri: /private/var/folders/bf/f_z6y6n959193x3csh_pcb0m0000gn/T/tmp.Zjp9bgzU/sam-app/hello_world runtime: python3.8 metadata: {} functions: ['HelloWorldFunction']

Fetching public.ecr.aws/sam/build-python3.8:latest Docker container image......
Mounting /private/var/folders/bf/f_z6y6n959193x3csh_pcb0m0000gn/T/tmp.Zjp9bgzU/sam-app/hello_world as /tmp/samcli/source:ro,delegated inside runtime container

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource

Expected result:

“Valid cache found” message on second run

❯ sam build -cu
Starting Build use cache
Starting Build inside a container
Cache is invalid, running build and copying resources to function build definition of 72cbac78-847c-4b54-9953-5fe0d300b926
Building codeuri: /private/var/folders/bf/f_z6y6n959193x3csh_pcb0m0000gn/T/tmp.Zjp9bgzU/sam-app/hello_world runtime: python3.8 metadata: {} functions: ['HelloWorldFunction']

Fetching public.ecr.aws/sam/build-python3.8:latest Docker container image......
Mounting /private/var/folders/bf/f_z6y6n959193x3csh_pcb0m0000gn/T/tmp.Zjp9bgzU/sam-app/hello_world as /tmp/samcli/source:ro,delegated inside runtime container

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
❯ sam build -cu
Starting Build use cache
Starting Build inside a container
Valid cache found, copying previously built resources from function build definition of 72cbac78-847c-4b54-9953-5fe0d300b926

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: macOS 11.3.1
sam --version: 1.23.0
AWS region: eu-west-1

debug.log

Issue Analytics

State:
Created 2 years ago
Reactions:4
Comments:5 (3 by maintainers)

Top GitHub Comments

3reactions

mndevecicommented, Jun 10, 2021

Thanks for reporting and thorough analysis @wnkz !

Indeed it is coming from invalid cache definition because our builders are adding additional environment variables before building and that was causing cache to be invalid all the time.

I raised the PR for the fix, and this should be resolved within next release after the PR is merged.

Thanks!

0reactions

github-actions[bot]commented, Jul 5, 2021

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

Top Results From Across the Web

Optimizing builds with cache management

The only way to force a rebuild is by making sure that a layer before it has changed, or by clearing the build...

Always have to clear cache in .env? - Stack Overflow

Solution: Only clear the cache: php artisan config:clear and php artisan cache:clear . Now you can upload to server and edit .env file...

Environment Variables | LanCache.NET

If you are using docker from the command line environment variables are specified using -e VARIABLE=VALUE format when you launch the container.

Build Environment — Apptainer User Guide 1.0 documentation

Apptainer will cache SIF container images generated from remote sources, ... to pass through the APPTAINER_CACHEDIR environment variable, if you set it.

Amazon ECS container agent configuration

The Amazon ECS container agent supports a number of configuration options, most of which should be set through environment variables.