Different result of `$?` when stepping over code PS ISE versus VSCode PS Extension

System Details

System Details Output

### VSCode version: 1.40.2 f359dd69833dd8800b54d458f6d37ab7c78df520 x64

### VSCode extensions:
aaron-bond.better-comments@2.0.5
austin.code-gnu-global@0.2.2
donjayamanne.githistory@0.4.10
DryWolf.dark-plus-plus-theme@0.0.9
dunstontc.dark-plus-syntax@0.0.159
eamodio.gitlens@10.2.0
esbenp.prettier-vscode@3.11.0
firefox-devtools.vscode-firefox-debug@2.5.5
HookyQR.beautify@1.5.0
idbartosz.darkpp-italic@0.1.3
jmrog.vscode-nuget-package-manager@1.1.6
johnpapa.vscode-peacock@3.2.0
karigari.chat@0.29.0
mikestead.dotenv@1.0.1
mitaki28.vscode-clang@0.2.3
ms-azuretools.vscode-docker@0.9.0
ms-vscode-remote.remote-wsl@0.40.3
ms-vscode.cpptools@0.26.2
ms-vscode.csharp@1.21.8
ms-vscode.powershell@2019.11.0
ms-vsliveshare.vsliveshare@1.0.1293
ms-vsliveshare.vsliveshare-audio@0.1.80
ms-vsliveshare.vsliveshare-pack@0.3.4
ms-vsts.team@1.161.0
msjsdiag.debugger-for-chrome@4.12.2
Shan.code-settings-sync@3.4.3
Valiantsin.operatormonodarktheme@0.7.1
vsciot-vscode.vscode-arduino@0.2.28
yzane.markdown-pdf@1.4.1
yzhang.markdown-all-in-one@2.5.1


### PSES version: 1.13.1.0

### PowerShell version:

Name                           Value
----                           -----
PSVersion                      5.1.18362.145
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.18362.145
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

I had these settings:

    "powershell.powerShellExePath": "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe",
    "powershell.enableProfileLoading": false,
    "powershell.debugging.createTemporaryIntegratedConsole": true

Issue Description

Following the 100 IAC example at https://github.com/starkfell/100DaysOfIaC/blob/master/articles/day.20.azure.cli.logging.build.pipes.redirects.exit.codes.win.md appears to yield the following inconsistencies within the Powershell ISE versus the VSCode PS extension. Example reproduced here:

Example Script

The script below is going to be the focus of today’s topic and is designed to fail on purpose.

$ShowGroup = az group show 2>&1

# Variation 1 - Check Exit Code of the previous command.
if ($?)
{
    Write-Output "[---success---] Azure CLI command ran as intended."
}
else
{
    Write-Output "[---fail------] Azure CLI command failed."
    Write-Output "[---fail------] $ShowGroup."
}

Expected Behaviour

When stepping over the lines of the example in Powershell ISE, $? evaluates as False. This is correct as per the 100 IAC example.

Actual Behaviour

When stepping over the lines of the example in VSCode, $? evaluates as True. This is incorrect as per the 100 IAC example.

The behaviour of both is the same when resume execution is used and breakpoints are set in the relevant if statement blocks.

Additional Info

The behaviour of $LASTEXITCODE in the example, when loading profile is disabled, appears to work normally. I would expect the example to use $LASTEXITCODE as a call out to the AZ CLI is equivalent (?) to calling out to a win32 executable.

The example may be wrong, but the discrepancy was making it difficult to debug when stepping over, so I believe the behaviour should be consistent with PS ISE.

[rjmholt]

as I assume $? already handles all of this for me.

Actually, $? just relies on $LASTEXITCODE too. There have been discussions about this in the PowerShell repo that I wasn’t able to find immediately, but basically there are applications (on Windows as well as on *nix) that write to stderr even when they succeed.

$?'s virtue is that it also captures cmdlet/function failure.

[rjmholt]

If the cause of this is the PSReadLine prompt or us executing things in the background to provide intellisense, then this may be very difficult behaviour to change.

Fundamentally, PowerShell is single threaded, but user interaction involves several concurrent phenomena. To provide this concurrency (so that the prompt can block while you get intellisense, etc), we must rehost the prompt and perform calls to things in PSReadLine for each prompt. When those calls succeed, PowerShell sets $?, overwriting your command’s original $? value.

It might be possible to use some reflection to preserve and reset this value while debugging, but I can make no promises there. I would highly recommend you use $LASTEXITCODE in contexts where a native utility is used.