Build errors should flow through Azure Pipelines to GitHub Checks annotations

GitHub Checks have the ability to annotate errors as related to specific lines of code. Arcade-based builds should take advantage of this.

There are several layers of logic required here:

Arcade

The build process must emit messages in such a way that Azure Pipelines can understand them.

The MSBuild and VSBuild pipelines tasks accomplish this by using a custom logger, Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.

Arcade launches builds using a script rather than those tasks, and it doesn’t seem that there’s an easy way to get access to the logger DLL from inside an arbitrary script.

Arcade could reimplement the logger–it’s a pretty straightforward mapping from the MSBuild BuildErrorEventArgs fields to the metadata required in the console message.

Pipelines

Pipelines then needs to transfer the error-message metadata into a Checks annotation. At the moment, this doesn’t appear to be fully functional.

For example, I introduced a compile error into a repo that uses the MSBuild task (and thus the custom logger). Pipelines correctly flags the error (https://dev.azure.com/dnceng/public/_build/results?buildId=94488), but the corresponding Check run says only that the overall build failed.

For this part, we’ll need to engage the Azure DevOps folks.

[jaredpar]

@chcosta if that’s the case then my vote is move code here. Reason being it’s just saner to manage. I’d rather add functionality to arcade, especially when it’s this straight forward, than yet another NuPkg (that will invetiably just cause me to swear a lot later).

[chcosta]

Azure DevOps says they’re ok if we want to move the code for the MSBuild Logger to an open source repo or publish / consume a NuGet package of it.