Feature Request: More functional and parallel processing functionallity

Summary of the new feature/enhancement

I currently have this block of assignments within my script:

$a = Get-Foo | ConvertTo-Json
$b = Get-Bar
$c = Get-BarFoo
$d = Get-FooBar
$e = Get-FooFoo
$f = Get-BarBar
$g = Get-FooUsing -Parameter $a
Do-SomethingThatChangesGlobalState

where none of the lines depend on the former ones. Currently there is no good way to tell powershell to do all these pipes in parallel and just store the result in the variable.

Proposed technical implementation details (optional)

I think about something like:

#pureFunctions
$a = Get-Foo | ConvertTo-Json
$b = Get-Bar
$c = Get-BarFoo
$d = Get-FooBar
$e = Get-FooFoo
$f = Get-BarBar
$g = Get-FooUsing -Parameter $a
barrier
Do-SomethingThatChangesGlobalState
barrier

Where #pureFunctions tells powershell that the script contains only pure functions and that it is allowed to swap lines as long as lines containing one of the variables as parameter is executed after it is defined. E.g. $g is always executed after $a, but no other guarantee on the execution order needs to be provided and powershell should automatically parallelize the pipelines. For commandlets and functions that do have side effects a barrier-keyword is required, so that they can be used. Optionally also Monads (like in e.g. Haskell) could be implemented to “warp” these.

Ideally, the #pureFunctions declaration should not only work for scripts, but also for scriptblocks, so that a functional entity could easily be integrated into modules and sequential scripts.

Please let me know what you think.

[vexx32]

I would… suggest writing it with less duplication:

$yamlJob = {
    param(
        [Parameter(Mandatory)]
        [ValidateNotNullOrEmpty()]
        [String]
        $PSPath
    )
    $a = Get-Content -Path $PSPath -Raw | ConvertFrom-Yaml
    @{"getContent" = $a } 
}

$apiJob = {    
    param(
        [Parameter(Mandatory)]
        [ValidateNotNullOrEmpty()]
        [String]
        $xxToken, 
        
        [Parameter(Mandatory)]
        [ValidateNotNullOrEmpty]
        [String]
        $xxAPIUrl,

        [Parameter(Mandatory)]
        [scriptblock]
        $ApiAction
    )
    $null = Connect-xxAPI -Token $xxToken -APIurl $xxAPIUrl
    & $ApiAction
}

$xxInventory = @(
    Start-ThreadJob -ScriptBlock $yamlJob -ArgumentList "$PSScriptRoot/file.yaml"
    Start-ThreadJob -ScriptBlock $apiJob -ArgumentList $xxToken, $xxAPIUrl, {
        @{"xxInventoryObjectListOfTypeA" = Get-xxObjectListOfTypeA }
    }
    Start-ThreadJob -ScriptBlock $apiJob -ArgumentList $xxToken, $xxAPIUrl, {
        @{"xxInventoryObjectListOfTypeB" = Get-xxObjectListOfTypeB }
    }
    Start-ThreadJob -ScriptBlock $apiJob -ArgumentList $xxToken, $xxAPIUrl, {
        @{"xxInventoryObjectListOfTypeC" = Get-xxObjectListOfTypeC }
    }
)

Wait-Job -Job $xxInventory

$joinedDictInventory = $xxInventory.Output | ForEach-Object -Begin {[Hashtable]$aa = @{}} -Process {
    foreach($element in ($_.Keys)) {
        $aa.Add($element,$_[$element])
    }
} -End {$aa}

[SeeminglyScience]

Maybe it’s just me being more optimistic than about this than you are, but I see value for this for libraries and to simplify code of complex operations as well as reducing the time it takes to debug issues with side effects.

I should clarify that my concern is more about the amount of community adoption rather than whether it would be useful to those who did adopt.

I’ve been wrong before though, so if you’re passionate about it I’d recommend drafting an RFC. I can’t really see a way to make these changes without pretty significant rewrites of PowerShell’s architecture, so an implementation proposal would be very helpful to move the conversation forward.