question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Failed to install update 2.277.1

See original GitHub issue

Describe the bug Starting multiple runners fails after downloading update 2.277.1

To Reproduce I’m using docker-compose to manage the self hosted Github runners. This worked fine, but after the most recent update some of the containers are stuck in an update loop. According to the logs, it seems that the 2.277.1 update is downloaded and installed twice.

Expected behavior The runner updates once and then listens for jobs

Runner Version and Platform

  • Runner 2.277.1
  • Ubuntu 18
  • Docker 20.10.3
  • Docker-compose 1.27.4

What’s not working?

runner_6  |
runner_6  | --------------------------------------------------------------------------------
runner_6  | |        ____ _ _   _   _       _          _        _   _                      |
runner_6  | |       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
runner_6  | |      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
runner_6  | |      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
runner_6  | |       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
runner_6  | |                                                                              |
runner_6  | |                       Self-hosted runner registration                        |
runner_6  | |                                                                              |
runner_6  | --------------------------------------------------------------------------------
runner_6  |
runner_6  | # Authentication
runner_6  |
runner_6  |
runner_6  | √ Connected to GitHub
runner_6  |
runner_6  | # Runner Registration
runner_6  |
runner_6  |
runner_6  |
runner_6  | √ Runner successfully added
runner_6  | √ Runner connection is good
runner_6  |
runner_6  | # Runner settings
runner_6  |
runner_6  |
runner_6  | √ Settings Saved.
runner_6  |
runner_6  |
runner_6  | √ Connected to GitHub
runner_6  |
runner_6  | 2021-02-18 16:33:28Z: Listening for Jobs
runner_6  | Runner update in progress, do not shutdown runner.
runner_6  | Downloading 2.277.1 runner
runner_6  | Waiting for current job finish running.
runner_6  | Generate and execute update script.
runner_6  | Runner will exit shortly for update, should back online within 10 seconds.
runner_6  |
runner_6  | --------------------------------------------------------------------------------
runner_6  | |        ____ _ _   _   _       _          _        _   _                      |
runner_6  | |       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
runner_6  | |      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
runner_6  | |      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
runner_6  | |       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
runner_6  | |                                                                              |
runner_6  | |                       Self-hosted runner registration                        |
runner_6  | |                                                                              |
runner_6  | --------------------------------------------------------------------------------
runner_6  | Cannot configure the runner because it is already configured. To reconfigure the runner, run 'config.cmd remove' or './config.sh remove' first.
runner_6  |
runner_6  | √ Connected to GitHub
runner_6  |
runner_6  | 2021-02-18 16:33:46Z: Listening for Jobs
runner_6  | Runner update in progress, do not shutdown runner.
runner_6  | Downloading 2.277.1 runner
runner_6  | Waiting for current job finish running.
runner_6  | Generate and execute update script.
runner_6  | Runner will exit shortly for update, should back online within 10 seconds.
runner-setup_runner_6 exited with code 0
runner_6  | ldd: ./bin/libcoreclr.so: No such file or directory
runner_6  | ldd: ./bin/System.Security.Cryptography.Native.OpenSsl.so: No such file or directory
runner_6  | ldd: ./bin/System.IO.Compression.Native.so: No such file or directory
runner_6  | ./config.sh: line 80: ./bin/Runner.Listener: No such file or directory
runner_6  | ./run.sh: line 27: /home/runner/actions-runner/bin/Runner.Listener: No such file or directory
runner-setup_runner_6 exited with code 127
runner_6  | ldd: ./bin/libcoreclr.so: No such file or directory
runner_6  | ldd: ./bin/System.Security.Cryptography.Native.OpenSsl.so: No such file or directory
runner_6  | ldd: ./bin/System.IO.Compression.Native.so: No such file or directory
runner_6  | ./config.sh: line 80: ./bin/Runner.Listener: No such file or directory
runner_6  | ./run.sh: line 27: /home/runner/actions-runner/bin/Runner.Listener: No such file or directory

Dockerfile:

# base
FROM ubuntu:18.04

# set the github runner version
ARG RUNNER_VERSION="2.276.1"

# update the base packages
RUN apt-get update -y && apt-get upgrade -y

# install python and the packages the your code depends on along with jq so we can parse JSON
# add additional packages as necessary
RUN apt-get install -y curl jq build-essential libssl-dev libffi-dev python3 python3-venv python3-dev python3-pip ca-certificates gnupg iputils-ping libicu-dev sudo mysql-client openssh-client

# Custom add user
RUN adduser --disabled-password --gecos '' runner \
  && usermod -aG sudo runner \
  && mkdir -m 777 -p /home/runner \
  && sed -i 's/%sudo\s.*/%sudo ALL=(ALL:ALL) NOPASSWD : ALL/g' /etc/sudoers

# cd into the user directory, download and unzip the github actions runner
RUN mkdir /home/runner/actions-runner
RUN curl -O -L https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
RUN mv actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz /home/runner/actions-runner/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
RUN tar xzf /home/runner/actions-runner/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz -C /home/runner/actions-runner

# install some additional dependencies
RUN chown -R runner ~runner && /home/runner/actions-runner/bin/installdependencies.sh

# copy over the start.sh script
COPY start.sh /home/runner/start.sh

# make the script executable
RUN chown -R runner ~runner /home/runner/start.sh
RUN chmod +x /home/runner/start.sh

# Fix for https://github.com/actions/runner/issues/945 - ./bin/libSystem.Security.Cryptography.Native.OpenSsl.so: No such file or directory
RUN sed -i -e 's/libSystem./System./g' /home/runner/actions-runner/config.sh

# since the config and run script for actions are not allowed to be run by root,
# set the user to "docker" so all subsequent commands are run as the docker user
USER runner

# set the entrypoint to the start.sh script
ENTRYPOINT ["/home/runner/start.sh"]

Start.sh:

#!/bin/bash

ORGANIZATION=$ORGANIZATION
ACCESS_TOKEN=$ACCESS_TOKEN
SERVER_NAME=$SERVER

REG_TOKEN=$(curl -sX POST -H "Authorization: token ${ACCESS_TOKEN}" https://api.github.com/orgs/${ORGANIZATION}/actions/runners/registration-token | jq .token --raw-output)

cd /home/runner/actions-runner

UUID=$(cat /proc/sys/kernel/random/uuid)
./config.sh --url https://github.com/${ORGANIZATION} --token ${REG_TOKEN} --name ${UUID} --labels self-hosted,linux,X64,${SERVER} --unattended

cleanup() {
    echo "Removing runner..."
    ./config.sh remove --name ${UUID} --token ${REG_TOKEN}
}

trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM

./run.sh & wait $!

So it seems like the runner fails to auto update and then crashes. This doesn’t happen all the time, so it is difficult for me to debug.

Issue Analytics

  • State:open
  • Created 3 years ago
  • Reactions:5
  • Comments:11 (3 by maintainers)

github_iconTop GitHub Comments

4reactions
jschlytercommented, Sep 23, 2021

I get the same on attempt to upgrade to v2.283.1:

./config.sh: line 80: ./bin/Runner.Listener: No such file or directory
./run.sh: line 27: /home/runner/actions-runner/bin/Runner.Listener: No such file or directory
ldd: ./bin/libcoreclr.so: No such file or directory
ldd: ./bin/System.Security.Cryptography.Native.OpenSsl.so: No such file or directory
ldd: ./bin/System.IO.Compression.Native.so: No such file or directory
3reactions
itslennycommented, Jul 28, 2021

I hit this exact same issue with version 2.279.0 today. Would be really swell if github didn’t force update breaking changes like this.

Anyone found a solution to this?

This is the output I’m getting.

2021-07-28 18:29:54Z: Listening for Jobs
Runner update in progress, do not shutdown runner.
Downloading 2.279.0 runner
Waiting for current job finish running.
Generate and execute update script.
Runner will exit shortly for update, should back online within 10 seconds.
ldd: ./bin/libcoreclr.so: No such file or directory
ldd: ./bin/System.Security.Cryptography.Native.OpenSsl.so: No such file or directory
ldd: ./bin/System.IO.Compression.Native.so: No such file or directory
./config.sh: line 80: ./bin/Runner.Listener: No such file or directory
Read more comments on GitHub >

github_iconTop Results From Across the Web

Upgrading to Jenkins LTS 2.277.x
Installation Wizard may show up on startup (JENKINS-65149). Jenkins 2.277.1 removes the Upgrade Wizard that was no longer operational after Jenkins 2.0 (pull ......
Read more >
Anyone have issues upgrading jenkins to new LTS 2.277.1
Hi all, I get this error when truing to upgrade from LTS 2.263.4: java.lang. ... the installed plugins when they upgrade to Jenkins...
Read more >
Jenkins plugin 6.5.3 does not work with Jenkins 2.277.1
Hello team, with the upgrade of Jenkins to the version 2.277.1 the RA plugin configuration in freestyle job does not work. The existing...
Read more >
CloudBees Jenkins Platform 2.277.1.2
After updating an instance, the setup wizard is displayed. Jenkins 2.277.1 removes the Upgrade Wizard that was no longer operational after Jenkins 2.0....
Read more >
Unable to update/install Jenkins plugins
Hi I'm not able to Update/Install any plugins in Jenkins getting this error below. Branch API Failure - java.io.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found