setup-buildx-action hangs with 99% CPU when running on latest Github Action Runner v2.285.0
See original GitHub issueBehaviour
Our runners were auto-updated this morning from v2.284.0 to v2.285.0 and began to hang on the Download and Install Buildx step:
 
Steps to reproduce this issue
- Upgrade your runner to version v2.285.0
- Run the docker/setup-buildx-action@v1in your workflow
- You should see heavy CPU utilization that did not exist in v2.284.0
 
Expected behaviour
The step should complete in under 10s per our typical experience
Actual behaviour
The step is hanging indefinitely
Configuration
- Repository URL (if public): private
- Build URL (if public): private
Note: all steps prior to the failing step pass as expected
name: Test Environment
on:
  pull_request:
    types: [labeled, reopened, synchronize, ready_for_review]
  workflow_dispatch:
env:
  HEAD_SHA: ${{ github.event.pull_request.head.sha }}
jobs:
  build:
    name: Build Image
    runs-on: self-hosted
    steps:
      - name: Set outputs
        id: vars
        run: |
          echo "::set-output name=head_sha::$(echo $PR_HEAD_SHA | cut -c1-7)"
        env:
          PR_HEAD_SHA: ${{ env.HEAD_SHA }}
      - name: Login to DockerHub
        uses: docker/login-action@v1
        with:
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: redacted
          role-to-assume: redacted
      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
      - name: Set up Docker Context for Buildx
        id: buildx-context
        run: |
          docker context create test-builder
      - name: Set up Docker Buildx
        id: buildx
        uses: docker/setup-buildx-action@v1
        with:
          endpoint: test-builder
      - name: Build and push Docker images
        uses: docker/build-push-action@v2.7.0
Logs
Download the log file of your build and attach it to this issue.
Our logs are littered with private, implementation details. Could you help me isolate which part of the logs you need for diagnosis?
2021-12-01T19:25:54.2902593Z Found online and idle self-hosted runner in the  current repository's enterprise account that matches the required labels: 'self-hosted'
2021-12-01T19:25:54.4048239Z Waiting for the runner to pick up this job...
2021-12-01T19:26:54.7363687Z Job was not picked up by the preferred runner, try requesting additional runners to run this job
2021-12-01T19:26:54.8282602Z Waiting for a self-hosted runner to pickup this job...
2021-12-01T19:32:40.3175224Z Job is about to start running on the runner: self-hosted-runner-group-6zjgq-67xpv (enterprise)
2021-12-01T19:32:44.8744015Z Current runner version: '2.285.0'
2021-12-01T19:32:44.8753670Z Runner name: 'self-hosted-runner-group-6zjgq-67xpv'
2021-12-01T19:32:44.8755541Z Runner group name: 'self-hosted'
2021-12-01T19:32:44.8758083Z Machine name: 'self-hosted-runner-group-6zjgq-67xpv'
...
2021-12-01T19:32:49.5211515Z ##[group]Download and install buildx
2021-12-01T19:32:49.5825561Z Downloading https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-amd64
2021-12-01T20:12:24.6445002Z ##[error]The operation was canceled.
2021-12-01T20:12:24.6564749Z Post job cleanup.
2021-12-01T20:12:24.7568406Z Post job cleanup.
2021-12-01T20:12:24.9055991Z Post job cleanup.
2021-12-01T20:12:24.9780442Z Post job cleanup.
2021-12-01T20:12:25.0473555Z [command]/usr/local/bin/docker logout 
2021-12-01T20:12:25.1428510Z Removing login credentials for https://index.docker.io/v1/
2021-12-01T20:12:25.1518891Z Cleaning up orphan processes
Issue Analytics
- State:
- Created 2 years ago
- Reactions:12
- Comments:61 (19 by maintainers)
 Top Results From Across the Web
Top Results From Across the Web
fs.copyFileSync hangs for file created within same ... - GitHub
setup-buildx-action hangs with 99 % CPU when running on latest Github Action Runner v2.285.0 docker/setup-buildx-action#117.
Read more >GitHub Actions step is hanging until timeout · Issue #1326
Describe the bug At some point, a GitHub Actions step stops producing output and hangs until the workflow is terminated by timeout.
Read more >fs.copyFileSync hangs for file created within same ... - GitHub
setup-buildx-action hangs with 99 % CPU when running on latest Github Action Runner v2.285.0 docker/setup-buildx-action#117.
Read more >The runner always in busy state even the job is finished. #700
I have self-hosted deployment in kubernetes. I had a deployment job ran failed, but the runner can't pickup new job, checked the runner...
Read more >A process hangs on Ubuntu sometimes #979 - GitHub
The problem is that Hugo hangs on runs-on: ubuntu-18.04 and 20.04 ... Does the Ubuntu VM of GitHub Actions runner has any limitation?...
Read more > Top Related Medium Post
Top Related Medium Post
No results found
 Top Related StackOverflow Question
Top Related StackOverflow Question
No results found
 Troubleshoot Live Code
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free Top Related Reddit Thread
Top Related Reddit Thread
No results found
 Top Related Hackernoon Post
Top Related Hackernoon Post
No results found
 Top Related Tweet
Top Related Tweet
No results found
 Top Related Dev.to Post
Top Related Dev.to Post
No results found
 Top Related Hashnode Post
Top Related Hashnode Post
No results found

OK, what solved the issue for us was pinning the version to 287.1 and disabling the auto update of the runner:
@crazy-max If we add the extra
Install Docker Buildxstep the pipeline finished successfully.