Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

action runner does not work with CentOS Stream 9

See original GitHub issue

Describe the bug I’m registering a CentOS Stream 9 VM as self hosted runner. I got the following error when I run command ./config.sh --url https://github.com/xxxx --token xxxx --name runner-centos-stream-9-large --labels centos-stream-9,large --ephemeral --disableupdate --unattended

Error: The SSL connection could not be established, see inner exception.

The latest version of ca-certificates-2020.2.50-94.el9.noarch.rpm is installed.

I also tried on CentOS Stream 8 VM with same version of runner, register works without error.

To Reproduce Steps to reproduce the behavior:

  1. Deploy CentOS Stream 9 VM from https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/BaseOS/x86_64/images/CentOS-Stream-GenericCloud-9-20220516.0.x86_64.qcow2
  2. Install dependencies: acl, lttng-ust, openssl-libs, krb5-libs, zlib, libicu
  3. Download latest runner from https://github.com/actions/runner/releases/download/v2.291.1/actions-runner-linux-x64-2.291.1.tar.gz and extract the installer file
  4. Run command: ./config.sh --url https://github.com/xxxx --token xxxx --name runner-centos-stream-9-large --labels centos-stream-9,large --ephemeral --disableupdate --unattended
  5. See the error

Expected behavior Register successful.

Runner Version and Platform

v2.291.1

OS of the machine running the runner?

CentOS Stream 9

What’s not working?

The SSL connection could not be established, see inner exception.

[2022-05-17 13:54:33Z ERR  ConfigurationManager] Failed to get tenant credentials -- Atempt: 1
[2022-05-17 13:54:33Z ERR  ConfigurationManager] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: NotSignatureValid
   at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CompleteHandshake(SslAuthenticationOptions sslAuthenticationOptions)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at GitHub.Runner.Listener.Configuration.ConfigurationManager.GetTenantCredential(String githubUrl, String githubToken, String runnerEvent)

Job Log Output

--------------------------------------------------------------------------------
|        ____ _ _   _   _       _          _        _   _                      |
|       / ___(_) |_| | | |_   _| |__      / \   ___| |_(_) ___  _ __  ___      |
|      | |  _| | __| |_| | | | | '_ \    / _ \ / __| __| |/ _ \| '_ \/ __|     |
|      | |_| | | |_|  _  | |_| | |_) |  / ___ \ (__| |_| | (_) | | | \__ \     |
|       \____|_|\__|_| |_|\__,_|_.__/  /_/   \_\___|\__|_|\___/|_| |_|___/     |
|                                                                              |
|                       Self-hosted runner registration                        |
|                                                                              |
--------------------------------------------------------------------------------

# Authentication

The SSL connection could not be established, see inner exception.

Runner and Worker’s Diagnostic Logs

[2022-05-17 13:54:33Z ERR  ConfigurationManager] Failed to get tenant credentials -- Atempt: 1
[2022-05-17 13:54:33Z ERR  ConfigurationManager] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: NotSignatureValid
   at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CompleteHandshake(SslAuthenticationOptions sslAuthenticationOptions)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at GitHub.Runner.Listener.Configuration.ConfigurationManager.GetTenantCredential(String githubUrl, String githubToken, String runnerEvent)

To debug this issue I also tried openssl s_client -connect github.com:443 to check system ca certificate and trust status. Here’s the output.

CONNECTED(00000003)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, CN = DigiCert TLS Hybrid ECC SHA384 2020 CA1
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "GitHub, Inc.", CN = github.com
verify return:1
---
Certificate chain
 0 s:C = US, ST = California, L = San Francisco, O = "GitHub, Inc.", CN = github.com
   xxxxx
 1 s:C = US, O = DigiCert Inc, CN = DigiCert TLS Hybrid ECC SHA384 2020 CA1
   xxxxx
---
Server certificate
-----BEGIN CERTIFICATE-----
xxxxx
-----END CERTIFICATE-----
subject=C = US, ST = California, L = San Francisco, O = "GitHub, Inc.", CN = github.com
issuer=C = US, O = DigiCert Inc, CN = DigiCert TLS Hybrid ECC SHA384 2020 CA1
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: ECDSA
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2805 bytes and written 378 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_128_GCM_SHA256
Server public key is 256 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_128_GCM_SHA256
    Session-ID: FCF1A7989FAB082EF5B9A285BA3070AE1447C1FD1346A6DBE12C7A5A856F230A
    Session-ID-ctx:
    Resumption PSK: C42947DD5081D1279D871EF9406409213F783228F020A219248AE564A1F4FF69
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 7c 32 2a f5 da 2e db be-7f 4d e8 a0 d7 0f bc ef   |2*......M......
    0010 - bc 22 b9 19 b5 61 b9 71-2a b6 14 67 07 48 f0 aa   ."...a.q*..g.H..

    Start Time: 1652840443
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_128_GCM_SHA256
    Session-ID: 91D77EC84BAC6E4CE4EFA5995F8500B90A0C6E4153CA74CF98A4B208F690FEA7
    Session-ID-ctx:
    Resumption PSK: 323AE0644E3A097D1BDD6F5E26B6612FA9FE6D4185ED2BE1F98E427EF62E8182
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 40 a4 38 a6 b1 e8 e5 61-2e 30 d0 02 66 37 1c e3   @.8....a.0..f7..
    0010 - f9 12 13 2f 54 a3 0b 84-2b e4 31 9d c1 fe 6c 69   .../T...+.1...li

    Start Time: 1652840443
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

6reactions
henrywangcommented, Jun 28, 2022

@martinpitt @TingluoHuang There’s a workaround for this issue. Enable SHA-1 in RHEL 9 and CentOS Stream 9 by following command: # update-crypto-policies --set DEFAULT:SHA1 But as we all know, SHA-1 is not considered secure any more.

1reaction
martinpittcommented, May 19, 2022

I tried this on a RHEL 9.1 nightly VM, which is by and large the same as CentOS 9 stream.

sudo dnf install -y acl lttng-ust openssl-libs krb5-libs zlib libicu
curl -L -O https://github.com/actions/runner/releases/download/v2.291.1/actions-runner-linux-x64-2.291.1.tar.gz
tar xf actions-runner-linux-x64-2.291.1.tar.gz
./config.sh --url https://github.com/xxxx --token xxxx --name runner-centos-stream-9-large --labels centos-stream-9,large --ephemeral --disableupdate --unattended

Note I literally used the xxx here – I didn’t set up any project or token, the error seems to happen before already.

This gets the same error. The relevant part seems to be this:

System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: NotSignatureValid

But unfortunately no details. Is this not using the system OpenSSL config?

Read more comments on GitHub >

github_iconTop Results From Across the Web

Stream 9 can't secure boot? : r/CentOS
I'm trying to install Stream 9 from a USB drive, but I'm getting a secure boot error: Invalid Signature Detected.
Read more >
Unable to run rsyslogd as non-root user on CentOS Stream 9
This seems to be an unfortunate change that forbids anyone from running rsyslogd without root privileges. The code is here
Read more >
Chapter 8. Known issues Red Hat Enterprise Linux 9
This part describes known issues in Red Hat Enterprise Linux 9.0. ... To work around this problem, do not run Anaconda on the...
Read more >
When Migrating to CentOS Stream Makes Sense
Learn why migrating to CentOS Stream may not be a straightforward process and feasible option for certain circumstances.
Read more >
Before You Get Mad About The CentOS Stream Change ...
The Red Hat contingent working on OpenStack needed something more stable than Fedora because they weren't interested in doing Operating System ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Required checks not passing for reusable workflows

Next page

Configuration tries to use the registration endpoint even when passed `--token <token>`