PSNativeCommandPreserveBytePipe hangs on piping to python scripts in Windows

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues.
• Refer to the FAQ.
• Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Steps to reproduce

I’m trying out the preview feature PSNativeCommandPreserveBytePipe, and it seems to work in most cases but I’ve noticed it hangs when piping to Python scripts in certain situations. I have both Python 3.9.7 and 3.11.0 installed and the standard python launcher. I have some simple python scripts:

repr.py, which should print the escaped binary contents of stdin:

import sys
print(repr(sys.stdin.buffer.read()).lstrip('b'))

echo.py, which just prints it’s arguments:

import sys
print(' '.join(sys.argv[1:]))

I also have cat.exe and echo.exe binaries from Cygwin 64-bit as well as WSL2 (Ubuntu).

The following commands hang with PSNativeCommandPreserveBytePipe enabled, until hitting Ctrl+C:

.\echo.py hello | .\repr.py
echo.exe hello | .\repr.py
wsl echo hello | .\repr.py
'hello' | cat.exe | .\repr.py
'hello' | wsl cat | .\repr.py
[byte[]](1, 2, 3) | cat.exe | .\repr.py
[byte[]](1, 2, 3) | wsl cat | .\repr.py

However, if I prefix the repr.py script with python.exe or py.exe (the python launcher) they work as expected:

> .\echo.py hello | py .\repr.py
'hello\r\n'
> [byte[]](1, 2, 3) | cat.exe | python .\repr.py
'\x01\x02\x03'

Also piping strings or bytes directly from Powershell to the un-prefixed script works:

> 'hello' | .\repr.py
'hello\r\n'
> [byte[]](1, 2, 3) | .\repr.py
'\x01\x02\x03'

So it seems that this only effects native-to-native commands involving the Python launcher launched implicitly via file association as the target of the pipe. I haven’t tried using other scripting languages to see if it effects other implicitly launched binaries. I confirmed that this effects both Python 3.9 and 3.11.

Also assoc and ftype show the .py file association as:

> cmd /c assoc .py
.py=Python.File
> cmd /c ftype Python.File
Python.File="C:\WINDOWS\py.exe" "%L" %*

Expected behavior

The expected behavior for the first 5 hanging commands is to print `'hello\r\n'`, and the last 2 to print `'\x01\x02\x03'`.

Actual behavior

The commands hang and nothing happens until I hit Ctrl+C, which stops the command with no output.

Running the commands with PSNativeCommandPreserveBytePipe disabled does not hang and shows the legacy behavior (bytes are passed as line-separated strings).

Error details

No response

Environment data

Name                           Value
----                           -----
PSVersion                      7.4.0-preview.4
PSEdition                      Core
GitCommitId                    7.4.0-preview.4
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

[rmccampbell]

I believe with the preview feature all native-to-native pipelines are treated as binary data, so no distinction is needed. Between Powershell commands and native binaries it depends on the type, so a byte array e.g. [byte[]](1,2,3) is passed as raw binary input. (There are some commands like Get-Content that can return byte arrays with the -AsByteStream flag).

[SeeminglyScience]

I can repro, thanks for the report @rmccampbell! Lookin’ into it