`OutOfBand` custom format definitions require `-Force` when using `Format-*` cmdlets

This is related to a longstanding issue with PowerShell that recently was brought up in a discussion on PowerShell/PowerShell-RFC#215.

Consider the following 10 object types:

System.Management.Automation.DebugRecord System.Management.Automation.InformationRecord System.Management.Automation.VerboseRecord System.Management.Automation.WarningRecord Deserialized.System.Management.Automation.DebugRecord Deserialized.System.Management.Automation.VerboseRecord Deserialized.System.Management.Automation.WarningRecord System.Exception System.Management.Automation.ErrorRecord System.Management.Automation.ScriptBlock

When you are working with any of those types as captured data (i.e. not as data that is streamed something other than standard output, but as something that you captured in a variable and are outputting in your terminal), if you pipe an instance of any of those types to Format-Table, Format-List, Format-Wide, or Format-Custom, you will still be presented with their default (custom) format unless you use the -Force parameter that is common to the Format-* cmdlets.

This behaviour, which has been in PowerShell since version 1, is unexpected. If you pipe data to a specific Format-* cmdlet, you expect that data to be shown in that format without having to -Force it.

The problem with OutOfBand is that the way PowerShell is programmed today, the -Force parameter is required to override the OutOfBand default output for a given type. That behaviour is the crux of this issue.

I’m currently investigating what can be done, if anything, to allow these types to use the requested format without requiring -Force.

Steps to reproduce

function Test-FormatScriptBlock {
    [CmdletBinding()]
    param()
    Write-Verbose -Verbose -Message 'This function outputs a bunch of OutOfBand types.'
    {Write-Output 'This is a script block'}
    Write-Warning 'This might not function quite the way you think'
    Get-Process -Id 1234321231
    Write-Debug -Debug -Message 'Debugging is fun!'
    {'This is another script block'}
}

function Test-FormatError {
    [CmdletBinding()]
    param()
    Get-Item -LiteralPath DoesNotExist -ErrorVariable myError -ErrorAction SilentlyContinue
    Write-Information -MessageData 'This outputs an error.' -InformationAction Continue
    $myError
    Write-Information -MessageData 'All done.' -InformationAction Continue
}

Test-FormatScriptBlock | Format-List *
Test-FormatError | Format-List *

Expected behavior

VERBOSE: This function outputs a bunch of OutOfBand types.

Attributes      : {}
File            :
IsFilter        : False
IsConfiguration : False
Module          :
StartPosition   : System.Management.Automation.PSToken
DebuggerHidden  : False
Id              : 012350e3-0d0f-4922-b9f2-eef17544e2a3
Ast             : {Write-Output 'This is a script block'}

WARNING: This might not function quite the way you think
Get-Process : Cannot find a process with the process identifier 1234321231.
At line:7 char:5
+     Get-Process -Id 1234321231
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (1234321231:Int32) [Get-Process], ProcessCommandException
+ FullyQualifiedErrorId : NoProcessFoundForGivenId,Microsoft.PowerShell.Commands.GetProcessCommand

DEBUG: Debugging is fun!
Attributes      : {}
File            :
IsFilter        : False
IsConfiguration : False
Module          :
StartPosition   : System.Management.Automation.PSToken
DebuggerHidden  : False
Id              : 3af7b1c9-ba29-4155-87cd-925f9108f53b
Ast             : {'This is another script block'}

This outputs an error.

PSMessageDetails      :
Exception             : System.Management.Automation.ItemNotFoundException: Cannot find path 'DoesNotExist' because it does not exist.
                           at System.Management.Automation.LocationGlobber.ResolveDriveQualifiedPath(String path, CmdletProviderContext context, Boolean allowNonexistingPaths, CmdletProvider& providerInstance) in
                        C:\Users\poshoholic\source\repos\PowerShell\src\System.Management.Automation\namespaces\LocationGlobber.cs:line 559
                           at System.Management.Automation.LocationGlobber.GetGlobbedMonadPathsFromMonadPath(String path, Boolean allowNonexistingPaths, CmdletProviderContext context, CmdletProvider& providerInstance) in
                        C:\Users\poshoholic\source\repos\PowerShell\src\System.Management.Automation\namespaces\LocationGlobber.cs:line 218
                           at System.Management.Automation.LocationGlobber.GetGlobbedProviderPathsFromMonadPath(String path, Boolean allowNonexistingPaths, CmdletProviderContext context, ProviderInfo& provider, CmdletProvider&
                        providerInstance) in C:\Users\poshoholic\source\repos\PowerShell\src\System.Management.Automation\namespaces\LocationGlobber.cs:line 779
                           at System.Management.Automation.SessionStateInternal.GetItem(String[] paths, CmdletProviderContext context) in
                        C:\Users\poshoholic\source\repos\PowerShell\src\System.Management.Automation\engine\SessionStateItem.cs:line 130
                           at System.Management.Automation.ItemCmdletProviderIntrinsics.Get(String path, CmdletProviderContext context) in
                        C:\Users\poshoholic\source\repos\PowerShell\src\System.Management.Automation\engine\ItemCmdletProviderInterfaces.cs:line 202
                           at Microsoft.PowerShell.Commands.GetItemCommand.ProcessRecord() in C:\Users\poshoholic\source\repos\PowerShell\src\Microsoft.PowerShell.Commands.Management\commands\management\Navigation.cs:line 1969
TargetObject          : DoesNotExist
CategoryInfo          : ObjectNotFound: (DoesNotExist:String) [Get-Item], ItemNotFoundException
FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemCommand
ErrorDetails          :
InvocationInfo        : System.Management.Automation.InvocationInfo
ScriptStackTrace      : at Test-FormatError, <No file>: line 4
                        at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo : {0, 1}

All done.

Actual behavior

VERBOSE: This function outputs a bunch of OutOfBand types.
Write-Output 'This is a script block'
WARNING: This might not function quite the way you think
Get-Process : Cannot find a process with the process identifier 1234321231.
At line:7 char:5
+     Get-Process -Id 1234321231
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (1234321231:Int32) [Get-Process], ProcessCommandException
+ FullyQualifiedErrorId : NoProcessFoundForGivenId,Microsoft.PowerShell.Commands.GetProcessCommand

DEBUG: Debugging is fun!
'This is another script block'

This outputs an error.
Get-Item : Cannot find path 'DoesNotExist' because it does not exist.
At line:4 char:5
+     Get-Item -LiteralPath DoesNotExist -ErrorVariable myError -ErrorA ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : ObjectNotFound: (DoesNotExist:String) [Get-Item], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemCommand

All done.

Environment data

Name                           Value
----                           -----
PSVersion                      7.0.0-preview.2
PSEdition                      Core
GitCommitId                    7.0.0-preview.2-70-g8234fbb04b437748b6782e9c45b0026c374ef12d
OS                             Microsoft Windows 10.0.17763
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

[KirkMunro]

FYI, I have opened PR #10430 early as a work in progress to solicit feedback on the approach I’ve started taking to resolve this issue in an expected way.

It would only be breaking if someone was piping some data of these types that was captured from standard output to a Format-* command without -Force and then converting that to string with Out-String and doing something with the string, but that’s an obscure scenario.

Initial tests with this approach have been very positive.

[KirkMunro]

I finished writing up a bunch of Pester tests for out of band formatting and added them to the PR today. With that and the earlier refactoring work complete, it’s now ready for review, so if you’re inclined to want to have a closer look at this and share feedback on what I’ve done, please visit #10430 and take a look.