Self-hosted runner Status switch to offline after logout

Hello,

We’re running a self-hosted runner on macOS M1 with the Monterey OS version.

We installed the service using svc.sh install and start it with svc.sh start. Staring the service working without any issues, and the runner status is Idle.

The problem starts after we close the login session (SSH or Screen share). After a few minutes, the status changed to offline. When we enter again to the runner, the status change to Idle.

Checking the runner logs shows this:

[2021-11-11 13:50:06Z INFO HostContext] Well known directory ‘Bin’: ‘/Users/runner/runner-four-mac/bin’ [2021-11-11 13:50:06Z INFO HostContext] Well known directory ‘Root’: ‘/Users/runner/runner-four-mac’ [2021-11-11 13:50:06Z INFO HostContext] Well known directory ‘Bin’: ‘/Users/runner/runner-four-mac/bin’ [2021-11-11 13:50:06Z INFO HostContext] Well known directory ‘Bin’: ‘/Users/runner/runner-four-mac/bin’ [2021-11-11 13:50:06Z INFO HostContext] Well known directory ‘Root’: ‘/Users/runner/runner-four-mac’ [2021-11-11 13:50:06Z INFO UnixUtil] Which: ‘chmod’ [2021-11-11 13:50:06Z INFO UnixUtil] Location: ‘/bin/chmod’ [2021-11-11 13:50:06Z INFO UnixUtil] Running /bin/chmod 755 “/Users/runner/runner-four-mac/svc.sh” [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Starting process: [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] File name: ‘/bin/chmod’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Arguments: ‘755 “/Users/runner/runner-four-mac/svc.sh”’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Working directory: ‘/Users/runner/runner-four-mac’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Require exit code zero: ‘True’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Encoding web name: ; code page: ‘’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Force kill process on cancellation: ‘False’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Redirected STDIN: ‘False’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Persist current code page: ‘False’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Keep redirected STDIN open: ‘False’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] High priority process: ‘False’ [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished. [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished. [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Process started with process id 3012, waiting for process exit. [2021-11-11 13:50:06Z INFO ProcessInvokerWrapper] Finished process 3012 with exit code 0, and elapsed time 00:00:00.0347568. [2021-11-11 13:50:06Z INFO Listener] Runner execution has finished with return code 0

Looks like the process stopped working after logging out.

More logs:

[2021-11-11 14:07:46Z ERR MessageListener] Catch exception during get next message. [2021-11-11 14:07:47Z ERR MessageListener] System.TimeoutException: The HTTP request timed out after 00:01:00. —> System.Threading.Tasks.TaskCanceledException: The operation was canceled. —> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. —> System.Net.Sockets.SocketException (89): Operation canceled — End of inner exception stack trace — at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.Security.SslStream.<FillBufferAsync>g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask1 task, Int32 min, Int32 initial) at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory1 buffer) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) — End of inner exception stack trace — at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, HttpConnectionPool connectionPool, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken) at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at GitHub.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) — End of inner exception stack trace — at GitHub.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at GitHub.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable1 queryParameters, Object userState, CancellationToken cancellationToken) at GitHub.Runner.Listener.MessageListener.GetNextMessageAsync(CancellationToken token) [2021-11-11 14:07:47Z ERR MessageListener] ##################################################### [2021-11-11 14:07:47Z ERR MessageListener] System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. ---> System.Net.Sockets.SocketException (89): Operation canceled --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.Security.SslStream.<FillBufferAsync>g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask1 task, Int32 min, Int32 initial) at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory1 buffer) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, HttpConnectionPool connectionPool, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken) at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at GitHub.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) [2021-11-11 14:07:47Z ERR MessageListener] ##################################################### [2021-11-11 14:07:47Z ERR MessageListener] System.IO.IOException: Unable to read data from the transport connection: Operation canceled. ---> System.Net.Sockets.SocketException (89): Operation canceled --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.Security.SslStream.<FillBufferAsync>g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask1 task, Int32 min, Int32 initial) at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory1 buffer) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) [2021-11-11 14:07:47Z ERR MessageListener] ##################################################### [2021-11-11 14:07:47Z ERR MessageListener] System.Net.Sockets.SocketException (89): Operation canceled [2021-11-11 14:07:47Z INFO MessageListener] Retriable exception: The HTTP request timed out after 00:01:00. [2021-11-11 14:07:47Z ERR Terminal] WRITE ERROR: 2021-11-11 14:07:47Z: Runner connect error: The HTTP request timed out after 00:01:00.. Retrying until reconnected. [2021-11-11 14:07:47Z INFO RunnerServer] Refresh MessageQueue VssConnection to get on a different AFD node. [2021-11-11 14:07:47Z INFO RunnerServer] Establish connection with 60 seconds timeout. [2021-11-11 14:07:47Z INFO GitHubActionsService] Starting operation Location.GetConnectionData [2021-11-11 14:07:47Z INFO GitHubActionsService] Finished operation Location.GetConnectionData [2021-11-11 14:07:48Z INFO MessageListener] Sleeping for 25.893 seconds before retrying. [2021-11-11 14:09:04Z INFO Terminal] WRITE LINE: 2021-11-11 14:09:04Z: Runner reconnected. [2021-11-11 14:25:49Z INFO MessageListener] No message retrieved from session 'bb49b50e-4338-46a4-821e-cd80c5997162' within last 30 minutes. [2021-11-11 14:27:53Z WARN GitHubActionsService] GET request to https://pipelines.actions.githubusercontent.com/XmnnSsvjBXnSuyMv6O9s0RlhrxuSWrvC22eYgickmCea1v50Uj/_apis/distributedtask/pools/1/messages?sessionId=bb49b50e-4338-46a4-821e-cd80c5997162 timed out after 60 seconds. [2021-11-11 14:27:53Z ERR MessageListener] Catch exception during get next message. [2021-11-11 14:27:53Z ERR MessageListener] System.TimeoutException: The HTTP request timed out after 00:01:00. ---> System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. ---> System.Net.Sockets.SocketException (89): Operation canceled --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.Security.SslStream.<FillBufferAsync>g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask1 task, Int32 min, Int32 initial) at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory1 buffer) at System.Net.Http.HttpConnection.FillAsync() at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, HttpConnectionPool connectionPool, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken) at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at GitHub.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) --- End of inner exception stack trace --- at GitHub.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at GitHub.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable1 queryParameters, Object userState, CancellationToken cancellationToken) at GitHub.Runner.Listener.MessageListener.GetNextMessageAsync(CancellationToken token) [2021-11-11 14:27:53Z ERR MessageListener] ##################################################### [2021-11-11 14:27:53Z ERR MessageListener] System.Threading.Tasks.TaskCanceledException: The operation was canceled. —> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. —> System.Net.Sockets.SocketException (89): Operation canceled — End of inner exception stack trace — at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.Security.SslStream.<FillBufferAsync>g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask1 task, Int32 min, Int32 initial) at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory1 buffer) at System.Net.Http.HttpConnection.FillAsync() at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) — End of inner exception stack trace — at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, HttpConnectionPool connectionPool, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken) at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at GitHub.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) [2021-11-11 14:27:53Z ERR MessageListener] ##################################################### [2021-11-11 14:27:53Z ERR MessageListener] System.IO.IOException: Unable to read data from the transport connection: Operation canceled. —> System.Net.Sockets.SocketException (89): Operation canceled — End of inner exception stack trace — at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.Security.SslStream.<FillBufferAsync>g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask1 task, Int32 min, Int32 initial) at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory1 buffer) at System.Net.Http.HttpConnection.FillAsync() at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) [2021-11-11 14:27:53Z ERR MessageListener] ##################################################### [2021-11-11 14:27:53Z ERR MessageListener] System.Net.Sockets.SocketException (89): Operation canceled [2021-11-11 14:27:53Z INFO MessageListener] Retriable exception: The HTTP request timed out after 00:01:00. [2021-11-11 14:27:53Z ERR Terminal] WRITE ERROR: 2021-11-11 14:27:53Z: Runner connect error: The HTTP request timed out after 00:01:00… Retrying until reconnected.

Stop retry on SessionConflictException after retried for 240 seconds. Failed to create session. The actions runner runner-four-mac already has an active session for owner runnertwo. Runner listener exited with error code 1 Runner listener exit with terminated error, stop the service, no retry needed. 2021-11-11 08:26:31Z: Runner connect error: The HTTP request timed out after 00:01:00… Retrying until reconnected. 2021-11-11 08:27:49Z: Runner reconnected. 2021-11-11 08:48:55Z: Runner connect error: The HTTP request timed out after 00:01:00… Retrying until reconnected. 2021-11-11 13:01:16Z: Runner reconnected. Shutting down runner listener Sending SIGINT to runner listener to stop Sending SIGKILL to runner listener Exiting… An error occurred: nodename nor servname provided, or not known Runner listener exited with error code 2 Runner listener exit with retryable error, re-launch runner in 5 seconds. Shutting down runner listener Sending SIGINT to runner listener to stop Sending SIGKILL to runner listener Shutting down runner listener Sending SIGINT to runner listener to stop Sending SIGKILL to runner listener .path=/opt/homebrew/opt/ruby@2.7/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin .path=/opt/homebrew/opt/ruby@2.7/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin Starting Runner listener with startup type: service Started listener process, pid: 1783 Started running service

Expected behavior Runner status stays ideal after logging out or restarting.

Runner Version and Platform

Runner is built for OSX (X64) - osx-x64. RuntimeInformation: Darwin 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:24 PDT root:xnu-8019.41.5~1/RELEASE_ARM64_T8101. Version: 2.284.0 Commit: 9027c154d05776c16f6df6c0345bc9ec20440ebe

[F-WRunTime]

This should be officially documented for the agent setup for Mac M1 runners. If this is an official setup necessity, it would be nice to have some official supported guidance on the Docs page for runners. I’ve been looking all over for some help on why my agent won’t stay online.

[zivbaz]

Thank you @TingluoHuang, any idea how can I do it? Should I edit svc.sh file?