Memory leak when foreach-object parallel
See original GitHub issueSteps to reproduce
$scriptThrottleLimit = 4
$scriptTimeoutLimit = 0
$ErrorActionPreference = Continue
$test = Get-Aduser -Filter *
$test.count
14483
$test.samaccountname | Foreach-Object -Parallel {
Write-Output $_
} -ThrottleLimit $scriptThrottleLimit -TimeoutSeconds $scriptTimeoutLimit
Expected behavior
works normally, process all object
Actual behavior
After processing some pack of objects, server goes to out of memory, then pwsh stop using cpu (0-1%) and stop processing pipe. Only stop process works. I tried different numbers in throttlelimit, no change, it just slow memory leak.
https://prnt.sc/ueti4c
http://prntscr.com/uetj45
i think it not clean after each of parallel instance
Environment data
Name Value
---- -----
PSVersion 7.0.3
PSEdition Core
GitCommitId 7.0.3
OS Microsoft Windows 10.0.14393
Platform Win32NT
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
WSManStackVersion 3.0
Issue Analytics
- State:
- Created 3 years ago
- Comments:9 (3 by maintainers)
Top Results From Across the Web
Memory usage powershell 7.03 Foreach-object Parallel
My solution was to move all of the code I had inside the forEach parallel into a function outside and above the forEach...
Read more >Huge memory consumption using ForEach-Object -Parallel
I've a function that updates the stats in databases using ForEach-Object -Parallel. When I run it, to loop through the stats in a...
Read more >PowerShell 7 ForEach-Object -Parallel Memory Consumption
I'm undertaking a project where one of of the tasks involves processing some 350000 XML files. If each file takes one second to...
Read more >Powershell 7's parallel ForEach-Object is mind blowing.
I just installed v7 yesterday and have been putting it through the paces to see what I can use it for by overhauling...
Read more >Curious Case of Parallel.ForEach with BlockingCollection
ForEach together with a BlockingCollection. The application was continuously eating up the memory until no more incoming data could be processed ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found

This version of PowerShell does not have the perf improvement for ForEach-Object -Parallel, and creates and destroys a runspace for each loop iteration (perf improvement in 7.1 preview reuses runspaces by default).
But I have tested both versions for leaks, and haven’t found any.
However, running scripts in parallel does use a lot of resources. Try adding
[System.GC]::Collect()to your loop to help ensure CLR recovers resources.I notice two things in your test repro above:
You store output from Get-ADUser, which may use a lot of resources depending on how large the returned objects are. Instead you could just stream directly to For-EachObject -Parallel.
Your loop script writes to output/console, which is a serial operation. So your repro expends a tremendous amount of resources to parallelize something that is immediately re-serialized, and essentially provides no benefit.
/cc @PaulHigin for information.