question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

PipelineVariable not available when using Foreach-Object -Parallel

See original GitHub issue

Steps to reproduce

$MyDict = [System.Collections.Concurrent.ConcurrentDictionary[string,object]]::new()
Get-Process -PipelineVariable Processinfo | ForEach-Object -Parallel {
    $dict = $using:MyDict
    $dict.TryAdd($Processinfo.id, $Processinfo) | Out-Null
}

$MyDict.Keys.Count
1

$MyDict.Keys

Expected behavior

$MyDict.Keys.Count
200
$MyDict.Keys
0
2
3
9
1480
...

Actual behavior

$MyDict.Keys.Count
1

$MyDict.Keys

Environment data

$PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.1.3
PSEdition                      Core
GitCommitId                    7.1.3
OS                             Microsoft Windows 10.0.17763
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

The variable defined as the pipeline variable is not available in the foreach-object script when the option -Parallel is used, However then -Parallel is not used, the variable can be used correctly

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:1
  • Comments:11 (7 by maintainers)

github_iconTop GitHub Comments

3reactions
PaulHigincommented, Mar 25, 2021

Pipeline variables are not supported for Foreach-Object -Parallel. The RFC and blog specify that variables can be passed in only via pipeline input $_ and with the $using: keyword.

...
The only variable that automatically appears in the parallel script block is the piped in object. Other variables are passed in using the $using: keyword.
...

But we may need to update the documentation to make this more clear. This may be something that can be considered for future releases.

1reaction
mklement0commented, Mar 25, 2021

Thanks for clarifying, @PaulHigin; documenting this will certainly help, but providing an appropriate error message is important too.

If we don’t want to introduce a specific error message for this scenario, the existing, catch-all message should be amended, along the lines of:

A Using variable can be used only with Start-Job, Start-ThreadJob, 
ForEach-Object -Parallel, except with pipeline variables,
and Invoke-Command when targeting remote computers.

Note that I’ve cut out the workflow reference, as it doesn’t apply to PS Core anymore, and that I’ve also added Start-ThreadJob.

Read more comments on GitHub >

github_iconTop Results From Across the Web

ForEach-Object - Module: Microsoft.PowerShell.Core
PipelineVariable common parameter variables are not supported in Foreach-Object -Parallel scenarios even with the $using: keyword. Example 17: Passing variables ...
Read more >
How can I use the -PipelineVariable parameter with ...
What I am finding, however, is that in my final ForEach-Object block the $_ variable (the PSSession object) doesn't always match up with...
Read more >
How to use ForEach-Object Parallel cmdlet in PowerShell
You can see in the above script that only $computers can be accessible inside the Foreach loop via $_ but to access the...
Read more >
PowerShell: Practical Use of PipelineVariable Common ...
Practical examples that show how the PipelineVariable parameter can be applied on day-to-day automation and reporting tasks. - PowerShell.
Read more >
Should I be using ".ForEach()" and ".Where()" built-in object ...
In several of the PowerShell scripts I have written, I have traditionally used the ForEach cmdlet to iterate through arrays/collections.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found