Logic App Standard - Slow Performance - Loops

Creating a simple workflow which has actions to create an array of x items, loop through the items and add the current time into another array variable the performance seems extremely slow/limited. I’ve amended multiple host settings which are mentioned to increase throughput (Such as Jobs.BackgroundJobs.NumWorkersPerProcessorCount) and also concurrency settings with no significant impact. When setting the for each loop concurrency to its max(50 concurrent iterations) it completes the workflow in the following approximate times:

500 items: 30 seconds 1000 items: 2+ minutes 5000 items: 30 minutes

I’d expect all of these to be less than a minute with some of the lower end being within milliseconds. There does not appear to be an issue with the underlying compute as the % CPU and Memory do not exceed 70% and the plan never scales even under load which feels like there is either limitations of the product, configuration restricting throughput or a potential bottleneck somewhere such as the associated storage account.

I know that there is a SplitOn() feature which can be used to separate workflows for processing but I have a requirement to extract 100,000’s of messages from a Service Bus and transform them into a single batch. I’m hoping to understand if there is an issue with configuration which can be updated to massively improve the throughput or whether the limitation is due to the product and if we need to switch to using Azure Functions.

An example workflow which can be used for testing is below:

{
    "definition": {
        "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
        "actions": {
            "Compose": {
                "inputs": "@variables('testArray')",
                "runAfter": {
                    "For_each": [
                        "Succeeded"
                    ]
                },
                "type": "Compose"
            },
            "Execute_JavaScript_Code": {
                "inputs": {
                    "code": "return Array.from({length: 5000}, (x, i) => i);"
                },
                "runAfter": {
                    "Initialize_variable": [
                        "Succeeded"
                    ]
                },
                "type": "JavaScriptCode"
            },
            "For_each": {
                "actions": {
                    "Append_to_array_variable": {
                        "inputs": {
                            "name": "testArray",
                            "value": "@{items('For_each')}: @{utcNow()}"
                        },
                        "runAfter": {},
                        "type": "AppendToArrayVariable"
                    }
                },
                "foreach": "@outputs('Execute_JavaScript_Code')",
                "runAfter": {
                    "Execute_JavaScript_Code": [
                        "Succeeded"
                    ]
                },
                "type": "Foreach"
            },
            "Initialize_variable": {
                "inputs": {
                    "variables": [
                        {
                            "name": "testArray",
                            "type": "array"
                        }
                    ]
                },
                "runAfter": {},
                "type": "InitializeVariable"
            }
        },
        "contentVersion": "1.0.0.0",
        "outputs": {},
        "triggers": {
            "Recurrence": {
                "recurrence": {
                    "frequency": "Day",
                    "interval": 15
                },
                "type": "Recurrence"
            }
        }
    },
    "kind": "Stateful"
}

[asmason]

Some great advice here. I had a workflow with nested foreach loops, it was just adding strings to an array variable and doing string concatenation. I had just 1 iteration of the outer foreach, then 100 iterations of the inner foreach. It took 5 minutes to run the workflow. So I replaced both of the foreach actions with Inline Javascript ones. Now my workflow takes 3 seconds. 😃

[github-actions[bot]]

This issue was closed because it has been inactive for 7 days since being marked as stale.