Throttling concurrent activities programmatically

I’m looking for a suitable pattern for throttling the number of activities that are processed concurrently. I have an orchestrator function that needs to execute a list of activities and I don’t want them all to start simultaneously but to be able to control the concurrency level through a parameter received by the function. I’m aware of the maxConcurrentActivityFunctions setting, but it operates at the host level and, as far as I know, cannot be changed programmatically.

My initial attempt was to try something similar to the code below, but it does not appear to work within an orchestrator.

    using (var throttle = new SemaphoreSlim(input.DegreeOfParallelism))
    {
        var results = input.Operations.Select(async operation =>
        {
            try
            {
                await throttle.WaitAsync();
                return context.CallActivityAsync<string>("ExecuteOperation", operation);
            }
            finally
            {
                throttle.Release();
            }
        });

        await Task.WhenAll(results);
   }

Any suggestions?

[f2bo]

I realized the code above did not preserve the results from each activity. I cleaned it up a bit and created an extension method that you can use it to call activities that return a TResult from an input list of objects of type T.

public async static Task<IEnumerable<TResult>> ThrottleActivities<TResult, T>(
    this DurableOrchestrationContext context,
    IEnumerable<T> activities,
    string functionName,
    int degreeOfParallelism)
{
    var runningActivities = new List<Task<TResult>>();
    foreach (var activity in activities)
    {
        var pendingOperations = runningActivities.Where(p => !p.IsCompleted);
        if (pendingOperations.Count() >= degreeOfParallelism)
        {
            await Task.WhenAny(pendingOperations);
        }

        var result = context.CallActivityAsync<TResult>(functionName, activity);
        runningActivities.Add(result);
    }

    var results = await Task.WhenAll(runningActivities);

    return results;
}

For example, to invoke activities that return strings for an input list of objects of type Operation and ensure that no more than 5 are started concurrently.

[FunctionName("ExecuteBatchJob")]
public static async Task ExecuteBatchJobAsync(
    [OrchestrationTrigger] DurableOrchestrationContext context,
    ILogger logger)
{
    var input = context.GetInput<BatchJob>();

    var results = await context.ThrottleActivities<string, Operation>(
        input.Operations,
        "ExecuteOperation",
        5);

    logger.LogDebug($"Completed all operations!");
}

[tmenier]

@cgillum Thanks for the explanation, that all makes perfect sense. I know this issue is long since closed but FWIW I think a function-level throttling feature like this would be a fantastic addition to the SDK. Thanks @f2bo for the code sample, this should work well for my scenario!