Build Push action failing with GHA cache on self-hosted runner

Troubleshooting

Behaviour

The action fails at the caching step.

The abort message is “Github actions is temporarily unavailable”, but status shows no issues with github actions. Additionally, other steps in the build process work just fine. This issue is still here after several days.

Also potentially relevant: I am using a self-hosted runner on pop-os

The relevant logs are here:

#20 exporting cache
#20 preparing build cache for export
#20 preparing build cache for export 44.4s done
#20 writing layer sha256:30e1b5776710e06922c72408b2d8e6ac56f1cf66b7d97697c8c28eafe4047ba8
#20 writing layer sha256:30e1b5776710e06922c72408b2d8e6ac56f1cf66b7d97697c8c28eafe4047ba8 0.5s done
#20 writing layer sha256:371803ae1c4092d5f537dcd5d83325e87d3f137ee2b5a827a9ad4c553c6dadde
#20 writing layer sha256:371803ae1c4092d5f537dcd5d83325e87d3f137ee2b5a827a9ad4c553c6dadde 0.2s done
#20 writing layer sha256:6603222a2d2bc0adbeb6f130df205a6d879ef0e45e2a6fb1362e5549eac7f5a1
#20 writing layer sha256:6603222a2d2bc0adbeb6f130df205a6d879ef0e45e2a6fb1362e5549eac7f5a1 0.2s done
#20 writing layer sha256:6e03d496f5213fdc7b85bafc814df0743309719ffa366c78da9b33f4ab19cfe4
#20 writing layer sha256:6e03d496f5213fdc7b85bafc814df0743309719ffa366c78da9b33f4ab19cfe4 0.2s done
#20 writing layer sha256:83098237b6d3febc7584c1f16076a32ac01def85b0d220ab46b6ebb2d6e7d4d4
#20 writing layer sha256:83098237b6d3febc7584c1f16076a32ac01def85b0d220ab46b6ebb2d6e7d4d4 0.0s done
#20 writing layer sha256:88a1d963e569e91cd91639d93f6df2bdb1e64b7cd394003a5d8bb54ff231c11e
#20 writing layer sha256:88a1d963e569e91cd91639d93f6df2bdb1e64b7cd394003a5d8bb54ff231c11e 0.2s done
#20 writing layer sha256:93bbf21dbbeec1bbdb76eb6bd4ff5a2bceae29fb2b8bc287a13b8572f9c23c03
#20 writing layer sha256:93bbf21dbbeec1bbdb76eb6bd4ff5a2bceae29fb2b8bc287a13b8572f9c23c03 60.2s done
#20 ERROR: error writing layer blob: failed to parse error response 503: { 'message': 'GitHub Actions is temporarily unavailable. Please visit https://www.githubstatus.com/ for the status of our services.', 'ref': 'Ref A: 96CC8A3C83314F12BEA57B8D983F48B6 Ref B: ASHEDGE1321 Ref C: 2021-12-08T15:48:22Z' }: invalid character '\'' looking for beginning of object key string
------
 > exporting cache:
------
error: failed to solve: error writing layer blob: failed to parse error response 503: { 'message': 'GitHub Actions is temporarily unavailable. Please visit https://www.githubstatus.com/ for the status of our services.', 'ref': 'Ref A: 96CC8A3C83314F12BEA57B8D983F48B6 Ref B: ASHEDGE1321 Ref C: 2021-12-08T15:48:22Z' }: invalid character '\'' looking for beginning of object key string
Error: buildx failed with: error: failed to solve: error writing layer blob: failed to parse error response 503: { 'message': 'GitHub Actions is temporarily unavailable. Please visit https://www.githubstatus.com/ for the status of our services.', 'ref': 'Ref A: 96CC8A3C83314F12BEA57B8D983F48B6 Ref B: ASHEDGE1321 Ref C: 2021-12-08T15:48:22Z' }: invalid character '\'' looking for beginning of object key string

Is there any issue with using gha cache in build-push-action on a self-hosted runner? If so, can we add some documentation of this breakage and how to work-around it? Maybe a better error message should be added as well.

Steps to reproduce this issue

1. Run action with GHA cache on ubuntu-latest
2. Verify that the action does not fail
3. Re-run action with GHA cache on [self-hosted, linux]
4. Verify that the action does fail

Expected behaviour

Action should not fail

Actual behaviour

Action failed

Configuration

• Repository URL (if public): private
• Build URL (if public): private

# action.yml
name: Test and Build
on:
  pull_request:
  push: 
    branches: [main]

jobs:
  build_and_test_and_push:
    runs-on: [self-hosted, linux]
    steps:
      - uses: actions/checkout@v2
        with:
          submodules: recursive

      - name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v1
        with:
          install: true
      -
        name: Login to DockerHub
        uses: docker/login-action@v1 
        with:
          username: ${{ secrets.DOCKER_USER }}
          password: ${{ secrets.DOCKER_PASS }}

      - name: Test Build
        uses: docker/build-push-action@master
        with:
          builder: ${{ steps.buildx.outputs.name }}
          context: .
          file: ${{ secrets.DOCKER_TEST_FILE }}
          push: false
          cache-from: type=gha
          cache-to: type=gha,mode=max

      - name: Build production image
        uses: docker/build-push-action@master
        with:
          builder: ${{ steps.buildx.outputs.name }}
          file: ${{ secrets.DOCKER_FILE }}
          context: .
          push: true
          tags: ${{ secrets.DOCKER_USER }}/${{ secrets.DOCKER_REPO }}:latest
          platforms: linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v6
          cache-from: type=gha
          cache-to: type=gha,mode=max

Logs

logs_119.zip

[jacobalberty]

An option to make it a warning or a fail could be a solution. I exclusively use this in public repos right now so a spike in usage isnt an issue for me its a minor inconvenience. I have to structure my builds right now so I only run my docker build at the end so a fail doesnt take out my build pipeline and even then it makes things like pull requests require more examination on my part to determine if it failed the tests or if it just failed the cache step.

If it was an option I could just set mine to not fail and just warn and things would go smoother for me. You could set yours to fail so you know to maybe delay new builds for a few days.

[crazy-max]

@krisztian-sala Looks like an issue with GitHub, see https://github.com/actions/cache/issues/698