Runner not using latest update, keeps trying to update to already installed update

Describe the bug Github runner tries to download and update to new 2.288.1 version from 2.287.1 for Windows. The download exits, so I have to execute run.cmd again. But when I run it again, it uses the old 2.287.1 and attempts to download 2.288.1 version again (it is already installed as I can see it in the Github Actions runner folder). Therefore, I cannot run Github Actions runner anymore.

To Reproduce Run run.cmd from runner version 2.287.1.

Expected behavior The Github runner should update and start running. Instead, it stops running after failing to update to 2.288.1. This is probably because 2.288.1 is already installed.

Runner Version and Platform

2.287.1 is being recognized, but 2.288.1 is installed as well.

OS of the machine running the runner? OSX/Windows/Linux/… Windows Server 2016 Datacenter 64-bit

What’s not working?

Output of run.cmd:

PS C:\azurestack-sdk-ci\runners\actions-runner-0> .\run.cmd

√ Connected to GitHub

Current runner version: '2.287.1'
2022-03-07 19:19:47Z: Listening for Jobs
Runner update in progress, do not shutdown runner.
Downloading 2.288.1 runner
PS C:\azurestack-sdk-ci\runners\actions-runner-0>

Job Log Output

No jobs were run.

Runner and Worker’s Diagnostic Logs

One of the errors in the Runner_…log file in the _diag/ log directory:

[2022-03-07 19:19:51Z INFO SelfUpdater] Version '2.288.1' of 'agent' package available in server.
[2022-03-07 19:19:51Z INFO SelfUpdater] Current running runner version is 2.287.1
[2022-03-07 19:19:51Z INFO SelfUpdater] An update is available.
[2022-03-07 19:19:51Z INFO Terminal] WRITE LINE: Runner update in progress, do not shutdown runner.
[2022-03-07 19:19:51Z INFO SelfUpdater] CloneDotnetRuntimeTime: 744ms
[2022-03-07 19:19:51Z INFO SelfUpdater] CloneExternalsTime: 104ms
[2022-03-07 19:19:51Z INFO SelfUpdater] RunnerPlatform: win-x64
[2022-03-07 19:19:51Z INFO Terminal] WRITE LINE: Downloading 2.288.1 runner
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] Sending CTRL_C to process 2164.
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] Successfully send CTRL_C to process 2164.
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] Waiting for process exit or 7.5 seconds after CTRL_C signal fired.
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] Ignore Ctrl+C to current process.
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] STDOUT/STDERR stream read finished.
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] Process exit successfully.
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] Process cancelled successfully through Ctrl+C/SIGINT.
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] Process Cancellation finished.
[2022-03-07 19:19:51Z INFO ProcessInvokerWrapper] Finished process 2164 with exit code -1073741510, and elapsed time 00:00:00.3848317.
[2022-03-07 19:19:51Z ERR  SelfUpdater] Fail to hash externals content: System.OperationCanceledException: The operation was canceled.
   at System.Threading.CancellationToken.ThrowOperationCanceledException()
   at GitHub.Runner.Sdk.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Channel`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
   at GitHub.Runner.Common.ProcessInvokerWrapper.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Channel`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
   at GitHub.Runner.Listener.SelfUpdater.HashFiles(String fileFolder, CancellationToken token)
   at GitHub.Runner.Listener.SelfUpdater.CloneAndCalculateAssetsHash(String dotnetRuntimeCloneDirectory, String externalsCloneDirectory, CancellationToken token)

Another error in the log:

[2022-03-07 19:19:55Z INFO SelfUpdater] Download latest runner and unzip into runner root.
[2022-03-07 19:19:55Z INFO Terminal] WRITE LINE: Waiting for current job finish running.
[2022-03-07 19:19:55Z INFO Terminal] WRITE LINE: Runner update process finished.
[2022-03-07 19:19:55Z INFO JobDispatcher] Shutting down JobDispatcher. Make sure all WorkerDispatcher has finished.
[2022-03-07 19:19:55Z ERR  Terminal] WRITE ERROR: An error occurred: The handle is invalid.
[2022-03-07 19:19:55Z ERR  Listener] System.IO.IOException: The handle is invalid.
   at System.ConsolePal.WindowsConsoleStream.Write(ReadOnlySpan`1 buffer)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.IO.StreamWriter.WriteLine(String value)
   at System.IO.TextWriter.SyncTextWriter.WriteLine(String value)
   at System.Console.WriteLine(String value)
   at GitHub.Runner.Common.Terminal.WriteLine(String line, Nullable`1 colorCode)
   at GitHub.Runner.Listener.SelfUpdater.UpdateRunnerUpdateStateAsync(String currentState)
   at GitHub.Runner.Listener.SelfUpdater.SelfUpdate(AgentRefreshMessage updateMessage, IJobDispatcher jobDispatcher, Boolean restartInteractiveRunner, CancellationToken token)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
   at GitHub.Runner.Listener.Runner.ExecuteCommand(CommandSettings command)
   at GitHub.Runner.Listener.Program.MainAsync(IHostContext context, String[] args)

[TheOnlyWei]

@ruvceskistefan The workaround provided by @ChristopherHX seems to be working, thanks for looking into this.

[ruvceskistefan]

@TheOnlyWei Great, thanks for response. Since this is a problem that is already known and solved in next version of the runner, I will close this issue. Feel free reopen it if you have the same problem.