Support Colorized Output in GitHub Actions

This may be a feature request more than an issue, but I’m out of my depth and having trouble finding where the breakdown occurs. Asking the experts! In response to a Twitter thread yesterday, I got some awesome feedback. One key piece of information is that GitHub Actions does support colorized console output (example). But I’ve never seen it in our dotnet test runs.

Here’s the monochromatic experience today:

We’re using xUnit which as far as I can tell only disables colorized output if -nocolor is passed to the console runner. That’s not the case, so I’m confused where in the pipe we’re dropping having better output. It will need a custom reporter (or at least detection) when GitHub Actions has a more full-fledged test results UI, but for now: colorized output would be a huge time saver and just generally make it a better experience.

Perhaps the fact that it’s wrapped in the GitHub Actions runner at all is the issue? There is #1762 related to being wrapped in another process - a result of IsOutputRedirected-style behavior breaking all? If it’s detection-based, it looks like the GITHUB_ACTIONS environmental variable is how it’s done elsewhere (example PR.

To get more info, I tried the /clp:ForceConsoleColor workaround for dotnet build (also currently monochrome in GitHub Actions). This did not work (example run here). We’d expect “Build Succeeded.” to be green as it is locally: …but it’s monochrome:

So…yeah I’m not sure what’s breaking down where and maybe it’s much more global that dotnet test. I just trying to even find where we can fix this. Is dotnet test eating the colorization? Or am I missing it and xUnit isn’t even trying to output it from a switch I’ve missed? Is is all dotnet * with Actions due to wrapping? Happy to close this issue and file it in the right place, and happy to assist in PR-ing towards a fix. Thanks for your time!

cc @bradwilson @clairernovotny @ethomson @livarcocc

[Tyrrrz]

I made a small proof of concept using GitHub Actions commands. This reports test failures to the GH Actions platform, letting you see them in the Annotations view and navigate to the exact/file that triggered it. This is not exactly the solution to “support colorized output” but may solve the underlying issue that GitHub Actions doesn’t show the list of failed tests nicely.

https://github.com/Tyrrrz/GitHubActionsTestLogger

https://github.com/Tyrrrz/GitHubActionsTestLogger/runs/525594126?check_suite_focus=true

Good enough?

[Tyrrrz]

I think introducing heuristics similar to Chalk’s is quite dangerous as they will become maintenance hell and will effectively turn dotnet into a bottleneck when new CIs are introduced/need to be added.

It makes more sense instead to add an environment variable, e.g. DOTNET_FORCE_CONSOLE_COLOR=1 or something that would cause ConsolePal to short-circuit and ignore the check for Console.IsOutputRedirected. Then the different CI providers or volunteers can ensure this environment variable is set on the agent. For example, with GitHub Actions, it could be set as part of the setup-dotnet action, similar to this.

Also, it sounds like this issue should be moved to a different repo?