Unable to run aws cli command from dockerfile

Troubleshooting:

I have the following docker file :

FROM node:14.17.0-slim as backend-assets

RUN apt-get update ***

# aws cli
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && unzip awscliv2.zip
RUN ./aws/install && aws --version

ARG AWS_ACCESS_KEY_ID
ARG AWS_SECRET_ACCESS_KEY
ARG AWS_SESSION_TOKEN

WORKDIR /root/backend

RUN aws s3 cp s3://XXXX assets --recursive
....

I used to build it locally by injecting AWS credentials ARGs as follow:

DOCKER_BUILDKIT=1 docker build --target backend-assets -t testing \
 --build-arg AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
 --build-arg AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
 --build-arg AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN --rm --progress=plain .

and then the following instruction works well so all files are downloaded from s3 bucket.

RUN aws s3 cp s3://XXXX assets --recursive

But when I tried to automate it in Github Actions using docker/build-push-action@v2 as follows:

- name: Build and Push backend-assets image
        uses: docker/build-push-action@v2
        with:  
          context: ./backend 
          push: true
          target: backend-assets
          tags: ${{ steps.login-ecr.outputs.registry }}/backend-assets:${{ steps.set-commit-sha.outputs.sha_short }}
          cache-from: type=registry,ref=${{ steps.login-ecr.outputs.registry }}/backend-assets:latest
          cache-to: type=inline
          build-args: |
            "AWS_ACCESS_KEY_ID=${{ env.AWS_ACCESS_KEY_ID }}"
            "AWS_SECRET_ACCESS_KEY=${{ env.AWS_SECRET_ACCESS_KEY }}"
            "AWS_SESSION_TOKEN=${{ env.AWS_SESSION_TOKEN }}"

the action raised the error below.

Behavior:

Expected behavior:

It should download files from s3 bucket.

Actual behavior:

error: failed to solve: process "/bin/sh -c aws s3 cp s3://XXXX assets --recursive --debug" did not complete successfully: exit code: 255
Error: buildx failed with: error: failed to solve: process "/bin/sh -c aws s3 cp s3://XXXX assets --recursive " did not complete successfully: exit code: 255

Logs:

This is the running action from Github logs:

/usr/bin/docker buildx build 
        --build-arg AWS_ACCESS_KEY_ID=*** \
        --build-arg AWS_SECRET_ACCESS_KEY=*** 
        --build-arg AWS_SESSION_TOKEN=XXXX 
        --tag ***.dkr.ecr.***.amazonaws.com/backend-assets:XXXX 
        --target backend-assets 
        --iidfile /tmp/docker-build-push-qsrsq2/iidfile 
        --metadata-file /tmp/docker-build-push-qsrsq2/metadata-file 
        --cache-from type=registry,ref=***.dkr.ecr.***.amazonaws.com/backend-assets:latest 
        --cache-to type=inline --push ./backend

Thanks in advance.

[crazy-max]

I think you’re missing the AWS_DEFAULT_REGION as I see in some blog posts out there:

• https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html
• https://andrewlock.net/packaging-cli-programs-into-docker-images-to-avoid-dependency-hell/
• https://lostechies.com/gabrielschenker/2016/09/21/easing-the-use-of-the-aws-cli/

It doesn’t work using secrets

The secret works as shown in https://github.com/docker/buildx/issues/809#issuecomment-947915521. It’s just the usage of the AWS command so I don’t think this is related to the action or the Dockerfile itself. Anyway I suggest to open a thread on StackOverflow about your issue.

[akramfstg]

Also ARG is not an env var so will not be visible by the AWS CLI command. You can do:

Aren’t they equivalent (during build?) i.e., both available as environment variable?

I already tried the docker file locally on my mac and it worked… why it cannot interpret that command?