Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Self hosted runner unusable after 2.287.1
See original GitHub issueDescribe the bug On our MacOS self-hosted runner 10.15.7 Catalina, we seem to have a repeated problem of the runner script becoming inoperable after a runner self-update occurs. The runner shows as idle in the settings for our repo and actively takes jobs but the jobs never start up and nothing is shown in stdout on the interactive runner. Looking at the logs the most recent update shows successful and all new jobs never get an entry in the _diag folder. I see successful logs leading up to the update. Here is the output we see on the interactive prompt:
2022-02-01 04:55:08Z: Job Run Frontend Benchmarks (macOS) completed with result: Succeeded
2022-02-01 05:15:12Z: Running job: Run Frontend Benchmarks (macOS)
2022-02-01 05:16:47Z: Job Run Frontend Benchmarks (macOS) completed with result: Succeeded
Runner update in progress, do not shutdown runner.
Downloading 2.287.1 runner
Waiting for current job finish running.
Generate and execute update script.
Runner will exit shortly for update, should be back online within 10 seconds.
Runner update process finished.
√ Connected to GitHub
Current runner version: '2.287.1'
2022-02-01 20:18:18Z: Listening for Jobs
[Process completed]
Currently, we simply restart the machine to fix the issue but this is just a band-aid. We have the “run.command” script run on boot-up for our machine. Is this not the preferred way to operate the runner?
Expected behavior The action runner service should continue functioning normally after an update.
Runner Version and Platform
Version of your runner? 2.287.1 and prior
OS of the machine running the runner? OSX/Windows/Linux/… 10.15.7 Catalina
What’s not working?
Seems the auto-update is borking our actions runner until a reboot occurs.
Job Log Output
There are no job logs on the self hosted runner but in github actions the message we see after job waits for some time is:
The self-hosted runner: Consoles-Mac-mini lost communication with the server. Verify the machine is running and has a healthy network connection. Anything in your workflow that terminates the runner process, starves it for CPU/Memory, or blocks its network access can cause this error.
Runner and Worker’s Diagnostic Logs
If applicable, add relevant diagnostic log information. Logs are located in the runner’s _diag folder. The runner logs are prefixed with Runner_ and the worker logs are prefixed with Worker_. Each job run correlates to a worker log. All sensitive information should already be masked out, but please double-check before pasting here.
Update logs:
[2022-02-01 12:18:14-4N] --------whoami--------
consolebench
[2022-02-01 12:18:14-4N] --------whoami--------
[2022-02-01 12:18:14-4N] Waiting for Runner.Listener (741) to complete
[2022-02-01 12:18:14-4N] Process 741 finished running
[2022-02-01 12:18:14-4N] Sleep 1 more second to make sure process exited
[2022-02-01 12:18:15-4N] Delete existing junction bin folder
[2022-02-01 12:18:15-4N] Delete existing junction externals folder
[2022-02-01 12:18:15-4N] Create junction bin folder
[2022-02-01 12:18:15-4N] Create junction externals folder
[2022-02-01 12:18:15-4N] Update succeed
[2022-02-01 12:18:15-4N] Rename /Users/consolebench/actions-runner/_diag/SelfUpdate-20220201-201814.log to be /Users/consolebench/actions-runner/_diag/SelfUpdate-20220201-201814.log.succeed
/Users/consolebench/actions-runner/_diag/SelfUpdate-20220201-201814.log -> /Users/consolebench/actions-runner/_diag/SelfUpdate-20220201-201814.log.succeed
[2022-02-01 12:18:15-4N] Restarting interactive runner
_diag/SelfUpdate-20220201-201814.log.succeed (END)
Runner logs:
[2022-02-01 20:18:16Z INFO HostContext] No proxy settings were found based on environmental variables (http_proxy/https_proxy/HTTP_PROXY/HTTPS_PROXY)
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Root': '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO HostContext] Well known config file 'Credentials': '/Users/consolebench/actions-runner/.credentials'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Root': '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO HostContext] Well known config file 'Runner': '/Users/consolebench/actions-runner/.runner'
[2022-02-01 20:18:16Z INFO Listener] Runner is built for OSX (X64) - osx-x64.
[2022-02-01 20:18:16Z INFO Listener] RuntimeInformation: Darwin 19.6.0 Darwin Kernel Version 19.6.0: Thu Jan 13 01:26:33 PST 2022; root:xnu-6153.141.51~3/RELEASE_X86_64.
[2022-02-01 20:18:16Z INFO Listener] Version: 2.287.1
[2022-02-01 20:18:16Z INFO Listener] Commit: b4e17c4fee3674494ffe88c5e06e42acb6788a37
[2022-02-01 20:18:16Z INFO Listener] Culture: en-US
[2022-02-01 20:18:16Z INFO Listener] UI Culture: en-US
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Root': '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO Listener] Validating directory permissions for: '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO CommandLineParser] Parse
[2022-02-01 20:18:16Z INFO CommandLineParser] Parsing 1 args
[2022-02-01 20:18:16Z INFO CommandLineParser] parsing argument
[2022-02-01 20:18:16Z INFO CommandLineParser] HasArgs: False
[2022-02-01 20:18:16Z INFO CommandLineParser] Adding Command: run
[2022-02-01 20:18:16Z INFO Listener] Arguments parsed
[2022-02-01 20:18:16Z INFO Runner] ExecuteCommand
[2022-02-01 20:18:16Z INFO ConfigurationStore] currentAssemblyLocation: /Users/consolebench/actions-runner/bin.2.287.1/Runner.Listener.dll
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO ConfigurationStore] binPath: /Users/consolebench/actions-runner/bin.2.287.1
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Root': '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO ConfigurationStore] RootFolder: /Users/consolebench/actions-runner
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Root': '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO HostContext] Well known config file 'Runner': '/Users/consolebench/actions-runner/.runner'
[2022-02-01 20:18:16Z INFO ConfigurationStore] ConfigFilePath: /Users/consolebench/actions-runner/.runner
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Root': '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO HostContext] Well known config file 'Credentials': '/Users/consolebench/actions-runner/.credentials'
[2022-02-01 20:18:16Z INFO ConfigurationStore] CredFilePath: /Users/consolebench/actions-runner/.credentials
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Root': '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO HostContext] Well known config file 'MigratedCredentials': '/Users/consolebench/actions-runner/.credentials_migrated'
[2022-02-01 20:18:16Z INFO ConfigurationStore] MigratedCredFilePath: /Users/consolebench/actions-runner/.credentials_migrated
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Root': '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO HostContext] Well known config file 'Service': '/Users/consolebench/actions-runner/.service'
[2022-02-01 20:18:16Z INFO ConfigurationStore] ServiceConfigFilePath: /Users/consolebench/actions-runner/.service
[2022-02-01 20:18:16Z INFO CommandSettings] Flag 'help': 'False'
[2022-02-01 20:18:16Z INFO CommandSettings] Flag 'version': 'False'
[2022-02-01 20:18:16Z INFO CommandSettings] Flag 'commit': 'False'
[2022-02-01 20:18:16Z INFO CommandSettings] Flag 'check': 'False'
[2022-02-01 20:18:16Z INFO CommandSettings] Command 'configure': 'False'
[2022-02-01 20:18:16Z INFO CommandSettings] Command 'remove': 'False'
[2022-02-01 20:18:16Z INFO CommandSettings] Command 'warmup': 'False'
[2022-02-01 20:18:16Z INFO ConfigurationManager] LoadSettings
[2022-02-01 20:18:16Z INFO ConfigurationStore] IsConfigured()
[2022-02-01 20:18:16Z INFO ConfigurationStore] IsConfigured: True
[2022-02-01 20:18:16Z INFO ConfigurationManager] Is configured: True
[2022-02-01 20:18:16Z INFO ConfigurationStore] Read setting file: 295 chars
[2022-02-01 20:18:16Z INFO ConfigurationManager] Settings Loaded
[2022-02-01 20:18:16Z INFO ConfigurationStore] IsServiceConfigured()
[2022-02-01 20:18:16Z INFO ConfigurationStore] IsServiceConfigured: False
[2022-02-01 20:18:16Z INFO CommandSettings] Command 'run': 'True'
[2022-02-01 20:18:16Z INFO ConfigurationStore] IsConfigured()
[2022-02-01 20:18:16Z INFO ConfigurationStore] IsConfigured: True
[2022-02-01 20:18:16Z INFO ConfigurationManager] Is configured: True
[2022-02-01 20:18:16Z INFO Runner] Could not parse the argument value '' for StartupType. Defaulting to Manual
[2022-02-01 20:18:16Z INFO Runner] Set runner startup type - Manual
[2022-02-01 20:18:16Z INFO CommandSettings] Flag 'once': 'False'
[2022-02-01 20:18:16Z INFO CommandSettings] Flag 'once': 'False'
[2022-02-01 20:18:16Z INFO Runner] RunAsync
[2022-02-01 20:18:16Z INFO ConfigurationManager] LoadSettings
[2022-02-01 20:18:16Z INFO ConfigurationStore] IsConfigured()
[2022-02-01 20:18:16Z INFO ConfigurationStore] IsConfigured: True
[2022-02-01 20:18:16Z INFO ConfigurationManager] Is configured: True
[2022-02-01 20:18:16Z INFO ConfigurationManager] Settings Loaded
[2022-02-01 20:18:16Z INFO MessageListener] {
"AgentId": 26,
"AgentName": "Consoles-Mac-mini",
"PoolId": 1,
"PoolName": "Default",
"ServerUrl": "https://pipelines.actions.githubusercontent.com/Knrc2HOOaAgOmct1wlLv6UBM3lrZfzQx0g8Ju84imjTmOSeiK3",
"GitHubUrl": "https://github.com/swift-nav/console_pp",
"WorkFolder": "_work"
}
[2022-02-01 20:18:16Z INFO MessageListener] Loading Credentials
[2022-02-01 20:18:16Z INFO ConfigurationStore] HasCredentials()
[2022-02-01 20:18:16Z INFO ConfigurationStore] stored True
[2022-02-01 20:18:16Z INFO CredentialManager] GetCredentialProvider
[2022-02-01 20:18:16Z INFO CredentialManager] Creating type OAuth
[2022-02-01 20:18:16Z INFO CredentialManager] Creating credential type: OAuth
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Bin': '/Users/consolebench/actions-runner/bin.2.287.1'
[2022-02-01 20:18:16Z INFO HostContext] Well known directory 'Root': '/Users/consolebench/actions-runner'
[2022-02-01 20:18:16Z INFO HostContext] Well known config file 'RSACredentials': '/Users/consolebench/actions-runner/.credentials_rsaparams'
[2022-02-01 20:18:16Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 20:18:16Z INFO MessageListener] Attempt to create session.
[2022-02-01 20:18:16Z INFO MessageListener] Connecting to the Runner Server...
[2022-02-01 20:18:16Z INFO RunnerServer] Establish connection with 100 seconds timeout.
[2022-02-01 20:18:16Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2022-02-01 20:18:16Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2022-02-01 20:18:16Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2022-02-01 20:18:16Z INFO RunnerServer] Establish connection with 60 seconds timeout.
[2022-02-01 20:18:16Z INFO GitHubActionsService] Starting operation Location.GetConnectionData
[2022-02-01 20:18:16Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2022-02-01 20:18:16Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2022-02-01 20:18:16Z INFO GitHubActionsService] Finished operation Location.GetConnectionData
[2022-02-01 20:18:17Z INFO MessageListener] VssConnection created
[2022-02-01 20:18:17Z INFO Terminal] WRITE LINE:
[2022-02-01 20:18:17Z INFO Terminal] WRITE LINE:
[2022-02-01 20:18:17Z WARN GitHubActionsService] Authentication failed with status code 401.
WWW-Authenticate: Bearer
Strict-Transport-Security: max-age=2592000
X-TFS-ProcessId: 9d09ab7b-fafd-45ef-be0e-882b77ffa10a
ActivityId: ea847dee-39fd-4e4b-a634-545a77af0a60
X-TFS-Session: 97b65b81-e160-40ac-bb49-ba6f8ec07d1e
X-VSS-E2EID: 61215d7f-285a-4e55-9184-e0af2e594e35
X-VSS-SenderDeploymentId: 2c974d96-2c30-cef5-eff2-3e0511a903a5
X-TFS-ServiceError: The+user+%27System%3aPublicAccess%3baaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%27+is+not+authorized+to+access+this+resource.
X-VSS-S2STargetService: 0000005A-0000-8888-8000-000000000000/visualstudio.com
X-Cache: CONFIG_NOCACHE
X-MSEdge-Ref: Ref A: 3793C7DC953240F6B6084472B1A0D41E Ref B: SJCEDGE0413 Ref C: 2022-02-01T20:18:17Z
Date: Tue, 01 Feb 2022 20:18:17 GMT
[2022-02-01 20:18:17Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 20:18:17Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 20:18:18Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2022-02-01 20:18:18Z INFO MessageListener] Session created.
[2022-02-01 20:18:18Z INFO Terminal] WRITE LINE: Current runner version: '2.287.1'
[2022-02-01 20:18:18Z INFO Terminal] WRITE LINE: 2022-02-01 20:18:18Z: Listening for Jobs
[2022-02-01 20:18:18Z INFO JobDispatcher] Set runner/worker IPC timeout to 30 seconds.
[2022-02-01 20:18:22Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 20:18:22Z INFO MessageListener] Message '915' received from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5'.
[2022-02-01 20:18:22Z INFO Runner] Received job message of length 43037 from service, with hash '46caec348932bec079906c1d565a67e240329cc2518d485769c8f5972b9a9027'
[2022-02-01 20:18:22Z INFO JobDispatcher] Job request 3736 for plan 741583ca-898f-4cdb-9551-a461b7773651 job cc20a10f-54c3-5fb7-3fc2-0f113a164aa6 received.
[2022-02-01 20:18:22Z INFO JobDispatcher] Pull OrchestrationId 741583ca-898f-4cdb-9551-a461b7773651.frontend_bench.macos from JWT claims
[2022-02-01 20:18:22Z INFO Terminal] WRITE LINE: 2022-02-01 20:18:22Z: Running job: Run Frontend Benchmarks (macOS)
[2022-02-01 20:28:14Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 20:28:14Z INFO MessageListener] Message '916' received from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5'.
[2022-02-01 20:28:14Z INFO JobDispatcher] Job cancellation request cc20a10f-54c3-5fb7-3fc2-0f113a164aa6 received, cancellation timeout 5 minutes.
[2022-02-01 20:48:50Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 20:48:50Z INFO MessageListener] Message '917' received from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5'.
[2022-02-01 20:48:50Z INFO Runner] Received job message of length 45110 from service, with hash '4732f4b5056cfce18e655947136a75f03e10da6de75ef7683cb1ef47c651f08d'
[2022-02-01 20:48:50Z INFO JobDispatcher] Job request 3746 for plan c9b0db51-d9de-4f09-89ac-e5046ff5edbc job cc20a10f-54c3-5fb7-3fc2-0f113a164aa6 received.
[2022-02-01 20:48:50Z INFO JobDispatcher] Pull OrchestrationId c9b0db51-d9de-4f09-89ac-e5046ff5edbc.frontend_bench.macos from JWT claims
[2022-02-01 20:48:50Z ERR JobDispatcher] Worker Dispatch failed with an exception for job request cc20a10f-54c3-5fb7-3fc2-0f113a164aa6.
[2022-02-01 20:48:50Z ERR JobDispatcher] System.IO.IOException: Input/output error
at System.ConsolePal.Write(SafeFileHandle fd, ReadOnlySpan`1 buffer, Boolean mayChangeCursorPosition)
at System.ConsolePal.UnixConsoleStream.Write(ReadOnlySpan`1 buffer)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.WriteLine(String value)
at System.IO.TextWriter.SyncTextWriter.WriteLine(String value)
at System.Console.WriteLine(String value)
at GitHub.Runner.Common.Terminal.WriteLine(String line, Nullable`1 colorCode)
at GitHub.Runner.Listener.JobDispatcher.RunAsync(AgentJobRequestMessage message, String orchestrationId, WorkerDispatcher previousJobDispatch, CancellationToken jobRequestCancellationToken, CancellationToken workerCancelTimeoutKillToken)
at GitHub.Runner.Listener.JobDispatcher.EnsureDispatchFinished(WorkerDispatcher jobDispatch, Boolean cancelRunningJob)
[2022-02-01 20:48:50Z INFO Terminal] WRITE LINE: 2022-02-01 20:48:50Z: Running job: Run Frontend Benchmarks (macOS)
[2022-02-01 21:13:04Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 21:13:04Z INFO MessageListener] Message '918' received from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5'.
[2022-02-01 21:13:04Z INFO Runner] Received job message of length 26118 from service, with hash '25b369d102013060350d3acfb40c99ce18c32c2d3079a647017a4959a1ad4db8'
[2022-02-01 21:13:04Z INFO JobDispatcher] Job request 3753 for plan 6028b9d7-67df-4f9f-9a02-9501ecea70ff job cc20a10f-54c3-5fb7-3fc2-0f113a164aa6 received.
[2022-02-01 21:13:04Z INFO JobDispatcher] Pull OrchestrationId 6028b9d7-67df-4f9f-9a02-9501ecea70ff.frontend_bench.macos from JWT claims
[2022-02-01 21:13:04Z ERR JobDispatcher] Worker Dispatch failed with an exception for job request cc20a10f-54c3-5fb7-3fc2-0f113a164aa6.
[2022-02-01 21:13:04Z ERR JobDispatcher] System.IO.IOException: Input/output error
at System.ConsolePal.Write(SafeFileHandle fd, ReadOnlySpan`1 buffer, Boolean mayChangeCursorPosition)
at System.ConsolePal.UnixConsoleStream.Write(ReadOnlySpan`1 buffer)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.WriteLine(String value)
at System.IO.TextWriter.SyncTextWriter.WriteLine(String value)
at System.Console.WriteLine(String value)
at GitHub.Runner.Common.Terminal.WriteLine(String line, Nullable`1 colorCode)
at GitHub.Runner.Listener.JobDispatcher.RunAsync(AgentJobRequestMessage message, String orchestrationId, WorkerDispatcher previousJobDispatch, CancellationToken jobRequestCancellationToken, CancellationToken workerCancelTimeoutKillToken)
at GitHub.Runner.Listener.JobDispatcher.EnsureDispatchFinished(WorkerDispatcher jobDispatch, Boolean cancelRunningJob)
[2022-02-01 21:13:04Z INFO Terminal] WRITE LINE: 2022-02-01 21:13:04Z: Running job: Run Frontend Benchmarks (macOS)
[2022-02-01 21:13:56Z WARN GitHubActionsService] Authentication failed with status code 401.
WWW-Authenticate: Bearer
Strict-Transport-Security: max-age=2592000
X-TFS-ProcessId: 792272df-b5fa-451f-a576-b23b3ebf93a6
ActivityId: 3a15b030-400a-4e92-bd12-abbf03f24923
X-TFS-Session: 97b65b81-e160-40ac-bb49-ba6f8ec07d1e
X-VSS-E2EID: b1d899c4-c186-4e4a-b2d2-a4204675ec3e
X-VSS-SenderDeploymentId: 2c974d96-2c30-cef5-eff2-3e0511a903a5
X-TFS-ServiceError: The+user+%27System%3aPublicAccess%3baaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%27+is+not+authorized+to+access+this+resource.
X-VSS-S2STargetService: 0000005A-0000-8888-8000-000000000000/visualstudio.com
X-Cache: CONFIG_NOCACHE
X-MSEdge-Ref: Ref A: 84ABDD0E1AE944AD99F153A610E4754D Ref B: SJCEDGE0413 Ref C: 2022-02-01T21:13:56Z
Date: Tue, 01 Feb 2022 21:13:56 GMT
[2022-02-01 21:13:56Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 21:13:56Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2022-02-01 21:43:09Z INFO MessageListener] No message retrieved from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5' within last 30 minutes.
[2022-02-01 21:52:29Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 21:52:29Z INFO MessageListener] Message '919' received from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5'.
[2022-02-01 21:52:29Z INFO Runner] Received job message of length 43045 from service, with hash '8dfd1416ba029f709947e524ea9848d77331ca2131db46d3b910acaf5ad1e5ca'
[2022-02-01 21:52:29Z INFO JobDispatcher] Job request 3760 for plan 873e010e-0772-4692-aef5-356a5cf66344 job cc20a10f-54c3-5fb7-3fc2-0f113a164aa6 received.
[2022-02-01 21:52:29Z INFO JobDispatcher] Pull OrchestrationId 873e010e-0772-4692-aef5-356a5cf66344.frontend_bench.macos from JWT claims
[2022-02-01 21:52:29Z ERR JobDispatcher] Worker Dispatch failed with an exception for job request cc20a10f-54c3-5fb7-3fc2-0f113a164aa6.
[2022-02-01 21:52:29Z ERR JobDispatcher] System.IO.IOException: Input/output error
at System.ConsolePal.Write(SafeFileHandle fd, ReadOnlySpan`1 buffer, Boolean mayChangeCursorPosition)
at System.ConsolePal.UnixConsoleStream.Write(ReadOnlySpan`1 buffer)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.WriteLine(String value)
at System.IO.TextWriter.SyncTextWriter.WriteLine(String value)
at System.Console.WriteLine(String value)
at GitHub.Runner.Common.Terminal.WriteLine(String line, Nullable`1 colorCode)
at GitHub.Runner.Listener.JobDispatcher.RunAsync(AgentJobRequestMessage message, String orchestrationId, WorkerDispatcher previousJobDispatch, CancellationToken jobRequestCancellationToken, CancellationToken workerCancelTimeoutKillToken)
at GitHub.Runner.Listener.JobDispatcher.EnsureDispatchFinished(WorkerDispatcher jobDispatch, Boolean cancelRunningJob)
[2022-02-01 21:52:29Z INFO Terminal] WRITE LINE: 2022-02-01 21:52:29Z: Running job: Run Frontend Benchmarks (macOS)
[2022-02-01 21:53:16Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 21:53:16Z INFO MessageListener] Message '920' received from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5'.
[2022-02-01 21:53:16Z INFO JobDispatcher] Job cancellation request cc20a10f-54c3-5fb7-3fc2-0f113a164aa6 received, cancellation timeout 5 minutes.
[2022-02-01 22:09:09Z WARN GitHubActionsService] Authentication failed with status code 401.
Transfer-Encoding: chunked
WWW-Authenticate: Bearer
Strict-Transport-Security: max-age=2592000
X-TFS-ProcessId: 09e30222-5b7f-4a7c-882d-5e9c7f34a54f
ActivityId: 225e74f2-4bb4-47f3-bf64-f724627c6d45
X-TFS-Session: 97b65b81-e160-40ac-bb49-ba6f8ec07d1e
X-VSS-E2EID: e20bca53-d5b3-4dac-8b1c-964ca0a65152
X-VSS-SenderDeploymentId: 2c974d96-2c30-cef5-eff2-3e0511a903a5
X-TFS-ServiceError: The+user+%27System%3aPublicAccess%3baaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%27+is+not+authorized+to+access+this+resource.
X-VSS-S2STargetService: 0000005A-0000-8888-8000-000000000000/visualstudio.com
X-Cache: CONFIG_NOCACHE
X-MSEdge-Ref: Ref A: 06D98ABE839A481D9119745088A1BC6D Ref B: SJCEDGE0413 Ref C: 2022-02-01T22:09:09Z
Date: Tue, 01 Feb 2022 22:09:08 GMT
[2022-02-01 22:09:09Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 22:09:09Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2022-02-01 22:23:21Z INFO MessageListener] No message retrieved from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5' within last 30 minutes.
[2022-02-01 22:47:42Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 22:47:42Z INFO MessageListener] Message '921' received from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5'.
[2022-02-01 22:47:42Z INFO Runner] Received job message of length 43045 from service, with hash '3224678332e1af287a3f4bf6dc8dfb97d4d631c41ea90e9e0ce9909c6307123c'
[2022-02-01 22:47:42Z INFO JobDispatcher] Job request 3781 for plan ab86222b-0921-4a13-959a-d6d0f34d739e job cc20a10f-54c3-5fb7-3fc2-0f113a164aa6 received.
[2022-02-01 22:47:42Z INFO JobDispatcher] Pull OrchestrationId ab86222b-0921-4a13-959a-d6d0f34d739e.frontend_bench.macos from JWT claims
[2022-02-01 22:47:42Z ERR JobDispatcher] Worker Dispatch failed with an exception for job request cc20a10f-54c3-5fb7-3fc2-0f113a164aa6.
[2022-02-01 22:47:42Z ERR JobDispatcher] System.IO.IOException: Input/output error
at System.ConsolePal.Write(SafeFileHandle fd, ReadOnlySpan`1 buffer, Boolean mayChangeCursorPosition)
at System.ConsolePal.UnixConsoleStream.Write(ReadOnlySpan`1 buffer)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.WriteLine(String value)
at System.IO.TextWriter.SyncTextWriter.WriteLine(String value)
at System.Console.WriteLine(String value)
at GitHub.Runner.Common.Terminal.WriteLine(String line, Nullable`1 colorCode)
at GitHub.Runner.Listener.JobDispatcher.RunAsync(AgentJobRequestMessage message, String orchestrationId, WorkerDispatcher previousJobDispatch, CancellationToken jobRequestCancellationToken, CancellationToken workerCancelTimeoutKillToken)
at GitHub.Runner.Listener.JobDispatcher.EnsureDispatchFinished(WorkerDispatcher jobDispatch, Boolean cancelRunningJob)
[2022-02-01 22:47:42Z INFO Terminal] WRITE LINE: 2022-02-01 22:47:42Z: Running job: Run Frontend Benchmarks (macOS)
[2022-02-01 23:04:25Z WARN GitHubActionsService] Authentication failed with status code 401.
WWW-Authenticate: Bearer
Strict-Transport-Security: max-age=2592000
X-TFS-ProcessId: 09e30222-5b7f-4a7c-882d-5e9c7f34a54f
ActivityId: 224c4d45-4bb4-47f3-bf64-f724627c6d45
X-TFS-Session: 97b65b81-e160-40ac-bb49-ba6f8ec07d1e
X-VSS-E2EID: 9da18e67-ce1a-4bd9-9140-3789a69b6738
X-VSS-SenderDeploymentId: 2c974d96-2c30-cef5-eff2-3e0511a903a5
X-TFS-ServiceError: The+user+%27System%3aPublicAccess%3baaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa%27+is+not+authorized+to+access+this+resource.
X-VSS-S2STargetService: 0000005A-0000-8888-8000-000000000000/visualstudio.com
X-Cache: CONFIG_NOCACHE
X-MSEdge-Ref: Ref A: 50B939D0003B4AB7B295AA21AEC25E2C Ref B: SJCEDGE0413 Ref C: 2022-02-01T23:04:25Z
Date: Tue, 01 Feb 2022 23:04:25 GMT
[2022-02-01 23:04:25Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 23:04:25Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2022-02-01 23:17:46Z INFO MessageListener] No message retrieved from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5' within last 30 minutes.
[2022-02-01 23:27:11Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 23:27:11Z INFO MessageListener] Message '922' received from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5'.
[2022-02-01 23:27:11Z INFO Runner] Received job message of length 26110 from service, with hash '49b89af88f8ab303ffbe66d94ec5e41b9b692490fb4dfdf000af37918c1100f6'
[2022-02-01 23:27:11Z INFO JobDispatcher] Job request 3811 for plan b49ecf50-76bf-43f7-90bb-b7d98ef401e0 job cc20a10f-54c3-5fb7-3fc2-0f113a164aa6 received.
[2022-02-01 23:27:11Z INFO JobDispatcher] Pull OrchestrationId b49ecf50-76bf-43f7-90bb-b7d98ef401e0.frontend_bench.macos from JWT claims
[2022-02-01 23:27:11Z ERR JobDispatcher] Worker Dispatch failed with an exception for job request cc20a10f-54c3-5fb7-3fc2-0f113a164aa6.
[2022-02-01 23:27:11Z ERR JobDispatcher] System.IO.IOException: Input/output error
at System.ConsolePal.Write(SafeFileHandle fd, ReadOnlySpan`1 buffer, Boolean mayChangeCursorPosition)
at System.ConsolePal.UnixConsoleStream.Write(ReadOnlySpan`1 buffer)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.WriteLine(String value)
at System.IO.TextWriter.SyncTextWriter.WriteLine(String value)
at System.Console.WriteLine(String value)
at GitHub.Runner.Common.Terminal.WriteLine(String line, Nullable`1 colorCode)
at GitHub.Runner.Listener.JobDispatcher.RunAsync(AgentJobRequestMessage message, String orchestrationId, WorkerDispatcher previousJobDispatch, CancellationToken jobRequestCancellationToken, CancellationToken workerCancelTimeoutKillToken)
at GitHub.Runner.Listener.JobDispatcher.EnsureDispatchFinished(WorkerDispatcher jobDispatch, Boolean cancelRunningJob)
[2022-02-01 23:27:11Z INFO Terminal] WRITE LINE: 2022-02-01 23:27:11Z: Running job: Run Frontend Benchmarks (macOS)
[2022-02-01 23:46:33Z INFO RSAFileKeyManager] Loading RSA key parameters from file /Users/consolebench/actions-runner/.credentials_rsaparams
[2022-02-01 23:46:33Z INFO MessageListener] Message '923' received from session '6cfaaf5c-a756-4c2a-a2ad-0d8283ff35d5'.
[2022-02-01 23:46:33Z INFO Runner] Received job message of length 25282 from service, with hash 'fc347a574bb440ac15f679cec5514fcf3530469dc725cdca1754ba74d30bdc11'
[2022-02-01 23:46:33Z INFO JobDispatcher] Job request 3815 for plan 73fc062e-2989-4028-afc8-8204903474ab job cc20a10f-54c3-5fb7-3fc2-0f113a164aa6 received.
[2022-02-01 23:46:33Z INFO JobDispatcher] Pull OrchestrationId 73fc062e-2989-4028-afc8-8204903474ab.frontend_bench.macos from JWT claims
[2022-02-01 23:46:33Z ERR JobDispatcher] Worker Dispatch failed with an exception for job request cc20a10f-54c3-5fb7-3fc2-0f113a164aa6.
[2022-02-01 23:46:33Z ERR JobDispatcher] System.IO.IOException: Input/output error
at System.ConsolePal.Write(SafeFileHandle fd, ReadOnlySpan`1 buffer, Boolean mayChangeCursorPosition)
at System.ConsolePal.UnixConsoleStream.Write(ReadOnlySpan`1 buffer)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.WriteLine(String value)
at System.IO.TextWriter.SyncTextWriter.WriteLine(String value)
at System.Console.WriteLine(String value)
at GitHub.Runner.Common.Terminal.WriteLine(String line, Nullable`1 colorCode)
at GitHub.Runner.Listener.JobDispatcher.RunAsync(AgentJobRequestMessage message, String orchestrationId, WorkerDispatcher previousJobDispatch, CancellationToken jobRequestCancellationToken, CancellationToken workerCancelTimeoutKillToken)
at GitHub.Runner.Listener.JobDispatcher.EnsureDispatchFinished(WorkerDispatcher jobDispatch, Boolean cancelRunningJob)
[2022-02-01 23:46:33Z INFO Terminal] WRITE LINE: 2022-02-01 23:46:33Z: Running job: Run Frontend Benchmarks (macOS)
(base) consolebench@Consoles-Mac-mini actions-runner %
Issue Analytics
- State:
- Created 2 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
Can you have a fallback option when GitHub self hosted ...
The issue is that if the self hosted runner is not available for some reason, and is not accessible for the time being,...
Read more >Self-hosted runners can now disable automatic updates
You now have more control over when your self-hosted runners perform software updates. If you specify the --disableupdate flag to the runner ...
Read more >Is the GitHub Actions self-hosted runner safe for Open Source?
GitHub warns against using self-hosted Actions runners for public repositories - but why? And are there alternatives?
Read more >GitHub Self hosted runners on AWS - part 1 - Fargate
Actions are the core re-usable bit that do the actual work. The actions will be executed on a runner, which is what we...
Read more >Issue when saving an artifact on self-hosted runner.
Summary. While using a self-hosted runner, the user faces an issue while trying to save an artifact. The runner log error message says:....
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
When our runner upgraded, it was able to upgrade successfully but it experienced this issue until we rebooted the system. I imagine the way to check would be after the new changes have been released, in say 2.287.2, and then observe the upgrade to whatever comes after, say 2.287.3. Unless the new changes would immediately effect the behavior of the auto-update process in which case we would know immediately in the next release. We can close this for now and I’ll add a new issue if I see the issue persist two releases from now.
Cheers
Hi @john-michaelburke,
Are you still experiencing this issue with upgrading to 2.287.1? Just to double-check if we should investigate this further.
If you are still seeing this issue, could you please provide the version from which the update occurred, so we can investigate exactly what happened just to be sure?