Failed to install update 2.277.1

Describe the bug Starting multiple runners fails after downloading update 2.277.1

To Reproduce I’m using docker-compose to manage the self hosted Github runners. This worked fine, but after the most recent update some of the containers are stuck in an update loop. According to the logs, it seems that the 2.277.1 update is downloaded and installed twice.

Expected behavior The runner updates once and then listens for jobs

Runner Version and Platform

• Runner 2.277.1
• Ubuntu 18
• Docker 20.10.3
• Docker-compose 1.27.4

What’s not working?

runner_6  |
runner_6  | --------------------------------------------------------------------------------
runner_6  | |        ____ _ _   _   _       _          _        _   _                      |
runner_6  | |       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
runner_6  | |      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
runner_6  | |      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
runner_6  | |       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
runner_6  | |                                                                              |
runner_6  | |                       Self-hosted runner registration                        |
runner_6  | |                                                                              |
runner_6  | --------------------------------------------------------------------------------
runner_6  |
runner_6  | # Authentication
runner_6  |
runner_6  |
runner_6  | √ Connected to GitHub
runner_6  |
runner_6  | # Runner Registration
runner_6  |
runner_6  |
runner_6  |
runner_6  | √ Runner successfully added
runner_6  | √ Runner connection is good
runner_6  |
runner_6  | # Runner settings
runner_6  |
runner_6  |
runner_6  | √ Settings Saved.
runner_6  |
runner_6  |
runner_6  | √ Connected to GitHub
runner_6  |
runner_6  | 2021-02-18 16:33:28Z: Listening for Jobs
runner_6  | Runner update in progress, do not shutdown runner.
runner_6  | Downloading 2.277.1 runner
runner_6  | Waiting for current job finish running.
runner_6  | Generate and execute update script.
runner_6  | Runner will exit shortly for update, should back online within 10 seconds.
runner_6  |
runner_6  | --------------------------------------------------------------------------------
runner_6  | |        ____ _ _   _   _       _          _        _   _                      |
runner_6  | |       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
runner_6  | |      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
runner_6  | |      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
runner_6  | |       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
runner_6  | |                                                                              |
runner_6  | |                       Self-hosted runner registration                        |
runner_6  | |                                                                              |
runner_6  | --------------------------------------------------------------------------------
runner_6  | Cannot configure the runner because it is already configured. To reconfigure the runner, run 'config.cmd remove' or './config.sh remove' first.
runner_6  |
runner_6  | √ Connected to GitHub
runner_6  |
runner_6  | 2021-02-18 16:33:46Z: Listening for Jobs
runner_6  | Runner update in progress, do not shutdown runner.
runner_6  | Downloading 2.277.1 runner
runner_6  | Waiting for current job finish running.
runner_6  | Generate and execute update script.
runner_6  | Runner will exit shortly for update, should back online within 10 seconds.
runner-setup_runner_6 exited with code 0
runner_6  | ldd: ./bin/libcoreclr.so: No such file or directory
runner_6  | ldd: ./bin/System.Security.Cryptography.Native.OpenSsl.so: No such file or directory
runner_6  | ldd: ./bin/System.IO.Compression.Native.so: No such file or directory
runner_6  | ./config.sh: line 80: ./bin/Runner.Listener: No such file or directory
runner_6  | ./run.sh: line 27: /home/runner/actions-runner/bin/Runner.Listener: No such file or directory
runner-setup_runner_6 exited with code 127
runner_6  | ldd: ./bin/libcoreclr.so: No such file or directory
runner_6  | ldd: ./bin/System.Security.Cryptography.Native.OpenSsl.so: No such file or directory
runner_6  | ldd: ./bin/System.IO.Compression.Native.so: No such file or directory
runner_6  | ./config.sh: line 80: ./bin/Runner.Listener: No such file or directory
runner_6  | ./run.sh: line 27: /home/runner/actions-runner/bin/Runner.Listener: No such file or directory

Dockerfile:

# base
FROM ubuntu:18.04

# set the github runner version
ARG RUNNER_VERSION="2.276.1"

# update the base packages
RUN apt-get update -y && apt-get upgrade -y

# install python and the packages the your code depends on along with jq so we can parse JSON
# add additional packages as necessary
RUN apt-get install -y curl jq build-essential libssl-dev libffi-dev python3 python3-venv python3-dev python3-pip ca-certificates gnupg iputils-ping libicu-dev sudo mysql-client openssh-client

# Custom add user
RUN adduser --disabled-password --gecos '' runner \
  && usermod -aG sudo runner \
  && mkdir -m 777 -p /home/runner \
  && sed -i 's/%sudo\s.*/%sudo ALL=(ALL:ALL) NOPASSWD : ALL/g' /etc/sudoers

# cd into the user directory, download and unzip the github actions runner
RUN mkdir /home/runner/actions-runner
RUN curl -O -L https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
RUN mv actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz /home/runner/actions-runner/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
RUN tar xzf /home/runner/actions-runner/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz -C /home/runner/actions-runner

# install some additional dependencies
RUN chown -R runner ~runner && /home/runner/actions-runner/bin/installdependencies.sh

# copy over the start.sh script
COPY start.sh /home/runner/start.sh

# make the script executable
RUN chown -R runner ~runner /home/runner/start.sh
RUN chmod +x /home/runner/start.sh

# Fix for https://github.com/actions/runner/issues/945 - ./bin/libSystem.Security.Cryptography.Native.OpenSsl.so: No such file or directory
RUN sed -i -e 's/libSystem./System./g' /home/runner/actions-runner/config.sh

# since the config and run script for actions are not allowed to be run by root,
# set the user to "docker" so all subsequent commands are run as the docker user
USER runner

# set the entrypoint to the start.sh script
ENTRYPOINT ["/home/runner/start.sh"]

Start.sh:

#!/bin/bash

ORGANIZATION=$ORGANIZATION
ACCESS_TOKEN=$ACCESS_TOKEN
SERVER_NAME=$SERVER

REG_TOKEN=$(curl -sX POST -H "Authorization: token ${ACCESS_TOKEN}" https://api.github.com/orgs/${ORGANIZATION}/actions/runners/registration-token | jq .token --raw-output)

cd /home/runner/actions-runner

UUID=$(cat /proc/sys/kernel/random/uuid)
./config.sh --url https://github.com/${ORGANIZATION} --token ${REG_TOKEN} --name ${UUID} --labels self-hosted,linux,X64,${SERVER} --unattended

cleanup() {
    echo "Removing runner..."
    ./config.sh remove --name ${UUID} --token ${REG_TOKEN}
}

trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM

./run.sh & wait $!

So it seems like the runner fails to auto update and then crashes. This doesn’t happen all the time, so it is difficult for me to debug.

[jschlyter]

I get the same on attempt to upgrade to v2.283.1:

./config.sh: line 80: ./bin/Runner.Listener: No such file or directory
./run.sh: line 27: /home/runner/actions-runner/bin/Runner.Listener: No such file or directory
ldd: ./bin/libcoreclr.so: No such file or directory
ldd: ./bin/System.Security.Cryptography.Native.OpenSsl.so: No such file or directory
ldd: ./bin/System.IO.Compression.Native.so: No such file or directory

[itslenny]

I hit this exact same issue with version 2.279.0 today. Would be really swell if github didn’t force update breaking changes like this.

Anyone found a solution to this?

This is the output I’m getting.

2021-07-28 18:29:54Z: Listening for Jobs
Runner update in progress, do not shutdown runner.
Downloading 2.279.0 runner
Waiting for current job finish running.
Generate and execute update script.
Runner will exit shortly for update, should back online within 10 seconds.
ldd: ./bin/libcoreclr.so: No such file or directory
ldd: ./bin/System.Security.Cryptography.Native.OpenSsl.so: No such file or directory
ldd: ./bin/System.IO.Compression.Native.so: No such file or directory
./config.sh: line 80: ./bin/Runner.Listener: No such file or directory