Feature Request: Allow writing to output streams from other threads

Summary of the new feature/enhancement

As a C# developer, when writing a PowerShell frontend to my .NET Standard or Core libraries, I would like to expose my library’s existing logging to PowerShell’s output streams - Verbose, Debug, Progress, etc. This is already easy to do with synchronous methods, but any async method causes threading problems that PowerShell cannot handle.

Currently, when Cmdlet.Write* is called from a background thread, an error message is thrown:

The WriteObject and WriteError methods cannot be called from outside the overrides of the BeginProcessing, ProcessRecord, and EndProcessing methods, and they can only be called from within the same thread. Validate that the cmdlet makes these calls correctly, or contact Microsoft Customer Support Services.

I would like the Cmdlet.Write* methods to allow writing on background threads, and handle marshalling data back to the primary thread as necessary.

Proposed technical implementation details (optional)

Here is a simple reproduction:

public interface ILogger
    {
        void Debug(string message);
        void Info(string message);
        // ...etc
    }

    public class PwshLogger : ILogger
    {
        public Cmdlet Cmdlet { get; }

        public PwshLogger(Cmdlet cmdlet) { this.Cmdlet = cmdlet; }

        // ILogger implementation
        public void Debug(string message) => this.Cmdlet.WriteDebug(message);
        public void Info(string message) => this.Cmdlet.WriteVerbose(message);
    }

    public class Widget
    {
        public ILogger Logger { get; set; }

        public int PerformTest()
        {
            Logger?.Info("Beginning test");
            Thread.Sleep(3000);
            Logger?.Info("Completing test");
            return 0;
        }

        public async Task<int> PerformTestAsync()
        {
            // Contrived example - normally, this would have some awaits in it
            var task = Task.Run(() => {
                Logger?.Info("Beginning async test");
                Thread.Sleep(3000);
                Logger?.Info("Completing async test");
            });
            task.GetAwaiter().GetResult();

            return 0;
        }
    }

    [Cmdlet(VerbsDiagnostic.Test, "Widget")]
    public class TestWidget : PSCmdlet
    {
        private PwshLogger _logger;
        private Widget _widget;

        protected override void BeginProcessing()
        {
            base.BeginProcessing();
            _logger = new PwshLogger(this);
            _widget = new Widget { Logger = _logger };
        }

        protected override void ProcessRecord()
        {
            base.ProcessRecord();

            // Works - run this cmdlet with -Verbose to see output
            _widget.PerformTest();

            // Does not work - throws the error detailed above
            Task.Run(() => _widget.PerformTestAsync()).GetAwaiter().GetResult();
        }
    }

Running the command provides this output:

PS> ipmo C:\Users\myUser\source\repos\PowerShellTest\PowerShellTest\bin\Debug\PowerShellTest.dll

PS> Test-Widget -Verbose
VERBOSE: Beginning test
VERBOSE: Completing test
Test-Widget : The WriteObject and WriteError methods cannot be called from outside the overrides of the BeginProcessing, ProcessRecord, and EndProcessing methods, and they can only be called from within the same thread. Validate that the cmdlet makes these calls correctly, or contact Microsoft Customer Support Services.
At line:1 char:1
+ Test-Widget -Verbose
+ ~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [Test-Widget], PSInvalidOperationException
+ FullyQualifiedErrorId : InvalidOperation,PowerShellTest.TestWidget

I have confirmed this behavior on Windows PowerShell 5.1, PowerShell 6.2.2, and PowerShell 7.0.0 preview 3.

I’ve seen a few questions about this, with various proposed solutions, but they all require the entire structure of the cmdlet class to be built around their solutions.

One could argue that a PowerShell frontend shouldn’t be exposing the internal logging from the module, but I’d argue that PowerShell’s native output streams are perfect for that sort of thing. The user can see no logging info if they want, or they can pass -Verbose and/or -Debug if they want detailed logging data displayed.

With the popularity of asynchronous programming, I believe it would be appropriate for PowerShell to handle this case gracefully in order to better integrate with the rest of .NET.

[SeeminglyScience]

At a glance it looks fine for what is there as far as I can tell.

Part of the problem with trying to tie async into PowerShell generically is that it’s a “pit of failure”. Meaning that there’s a lot of random engine specific knowledge you have to know in order to do it safely. PowerShell is designed from the ground up to store all of it’s state in a thread static way. The exception that gets thrown when trying to write from a different thread is actually the kind way this manifests.

Even something as simple as calling this.MyInvocation.MyCommand.Parameters from a different thread can cause dead locks and/or state corruption. The AsyncCmdlet used internally by the PackageManagement module had this issue for example.

While a lot of things may end up working fine, a seemingly small change can have huge consequences due to this architecture. Basically, if you’re on a different thread, you should just straight up never touch the PSCmdlet instance at all unless you have a deep understanding of how the engine works internally.

Personally, I’d like to see a utility class that makes this sort of thread communication easier but I fear tying it into a subclass of PSCmdlet may set the wrong expectation. Even if it was it’s own standalone class I’d be very hesitant to suggest it be added to SMA.

[BrucePay]

@replicaJunction I’m not sure I understand the motivation here. It sounds like you want to allow random threads to start writing error/verbose/information messages mixed in with what the user is actually doing. That sound’s like a very confusing experience to me. On the other hand, if the goal is to support in-band user notification of asynchronous events, that is more interesting but might lead to a different approach.