Runner Observability

Prerequisites

• You use GitHub Enterprise Server
• Naturally, you use Self-Hosted Runners
• You embrace DevOps, meaning you give the teams free reign over the runners activity

Nature of problem Assuming you have (like us) over 100 developers, dozens or hundreds of workflows. All share the same self-hosted runner(s). You have no oversight, who highjacks the runners. Highjack means hogging any form of resouce:

• Runtime
• Upload volume
• Log volume

Describe the enhancement The cleanest enhancement would be a form of extension hooks. Upon job start a hook in some form gets called, within this hook you could then define your own actions. Maybe something in style of Swizzling where the native hook does nothing (or a console log) while you can swizzle the component to add your own action. Upon completion another hook gets called with which you can then complete your observability.

Code Snippet

Some pseudo code. Given that the runner is .NET code it would not look like that, I just come from the TS world.

function onInit(flowId: UUID, runner: UUID, context: GitHubContext){
  infos = composeInfos(args);
  prometheus.pushgateway.push(infos);
}

function onInit(flowId: UUID, runner: UUID, context: GitHubContext, duration: number, uploadedBytes: number, loggedLines: number){
  infos = composeInfos(args);
  prometheus.pushgateway.push(infos);
}

Additional information It might be that this concept already exists, but then its just not documented or not findable.

Also, I once saw a /timing API but I can not find it anymore, seems to have been removed.

Clearly, when enterprises start to adopt Actions the demand for observability will raise. Are we alone? 🛸

[jbergstroem]

Other metrics that would make sense is for instance queue length. In gitlab land there are excellent ways of getting observability out of the runner via prometheus exporters. I wish the github runner took a similar approach.

[thboop]

We’ve shipped a beta of this functionality in 2.289.1, please try it out and provide any feedback you have on the adr!