Image not cached using Docker caching example from docs

Behaviour

Using the example workflow from the docs to build with a buildx local cache directory that is then kept in the Github Actions cache results in Docker layers still being downloaded by buildx. The workflow’s “Cache Docker Layers” step successfully restores the cache once it is warmed up, but the “Build and push” step shows layers being downloaded.

Steps to reproduce this issue

1. Create a new repo with a simple Dockerfile.
2. Add a Github Actions workflow using the example workflow from the docs.
3. Push an update to trigger a build and warm the cache.
4. Push another update to trigger another build.

Expected behaviour

The base image specified in the Dockerfile should be in buildx’s cache and the buildx logs should simply show “CACHED” after processing the FROM line.

Actual behaviour

The logs show “CACHED” but then show the progress lines of various layers of the base image being downloaded.

Configuration

• Repository with example: https://github.com/mn-git-hub/actions-cache-test
• Build with warmed up cache that shows the downloading: https://github.com/mn-git-hub/actions-cache-test/actions/runs/1758301913

The only change from the example workflow config is that push is disabled. I tried with push enabled but hit an auth issue with Dockerhub which is beyond the scope of this issue. Also, even in that build, the cache issue was still present.

# from https://github.com/docker/build-push-action/blob/master/docs/advanced/cache.md
name: ci

on:
  push:
    branches:
      - 'main'

jobs:
  docker:
    runs-on: ubuntu-latest
    steps:
      -
        name: Checkout
        uses: actions/checkout@v2
      -
        name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1
      -
        name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-buildx-
      -
        name: Login to DockerHub
        uses: docker/login-action@v1 
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}
      -
        name: Build and push
        uses: docker/build-push-action@v2
        with:
          context: .
          push: false
          tags: user/app:latest
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max
      -
        # Temp fix
        # https://github.com/docker/build-push-action/issues/252
        # https://github.com/moby/buildkit/issues/1896
        name: Move cache
        run: |
          rm -rf /tmp/.buildx-cache
          mv /tmp/.buildx-cache-new /tmp/.buildx-cache

Logs

1_docker.txt

[mnoble-vody]

I see. It ends up taking a significant amount of time processing the cached blobs, to the point where it is faster to not using the cache. Any ideas on why this would be going so slowly? This is running on the Github Actions hosted runners.

[tonistiigi]

Are you aware of this one

Yes, I remember this was reported on some issue. But I don’t know what conditions cause it.