WaitForOrchestrationAsync gets stuck in Running even though it finished?

Hi. I am trying to evaluate DTF to replace an in-house distributed task implementation, and while its looking great, I am having what seems to be a fairly fundamental issue (so probably I am doing something wrong 😉.

What I am seeing is that orchestrations execute just fine, but stay stuck in “Running” state. I have included a small example below. In this particular example, the “Done waiting” message is printed, but WaitForOrchestrationAsync never ends (and if I change the code to poll the state, it stays in “Running”. If I remove the await inside the orchestration it works fine but that is kind of the whole point of DTF 😃

This simple example is derived from a bigger example with the same problems (where the orchestration is only await’ing scheduled tasks).

var service = new LocalOrchestrationService();
await service.CreateIfNotExistsAsync();
await service.StartAsync();

var worker = new TaskHubWorker(service);
worker.AddTaskOrchestrations(typeof(UpperCaseOrchestrator));
await worker.StartAsync();

var client = new TaskHubClient(service);
var instance = await client.CreateOrchestrationInstanceAsync(typeof(UpperCaseOrchestrator), "hello world");
    
var state = await client.WaitForOrchestrationAsync(instance, TimeSpan.FromMilliseconds(100000));
Console.WriteLine($"Output: {state.Output}");

await worker.StopAsync(true);

public class UpperCaseOrchestrator : TaskOrchestration<string, string>
{
    public override async Task<string> RunTask(OrchestrationContext context, string input)
    {
        Console.WriteLine($"Input: {input}");
        await Task.Delay(100).ConfigureAwait(false);
        Console.WriteLine($"Done waiting");

        return input.ToUpperInvariant();
    }
}

My csproj file:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>net6.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
    </PropertyGroup>

    <ItemGroup>
      <PackageReference Include="Microsoft.Azure.DurableTask.Emulator" Version="2.5.4" />
      <PackageReference Include="Microsoft.Azure.DurableTask.Core" Version="2.5.6" />
    </ItemGroup>

</Project>

(I am using .net 6.0 rc1 here, but in the bigger project I am using netcoreapp3.1)

Hope something can point me in the right direction 😃

EDIT: Whether ConfigureAwait is used in the orchestration does not make any difference.

[petertiedemann]

Thanks. I am just so used to writing ConfigureAwait(false) that i don’t even think about it 😃

Looking forward to continuing trying this out, the ability to work with remote long running tasks using async await is really nice.

[cgillum]

Yes, ConfigureAwait(false) creates the exact same problem as Task.Delay. The orchestration scheduler gets confused if a task continuation is scheduled on any thread but the main execution thread, resulting in problems like this. We are able to detect it if you make calls to other context operations after awaiting, but it looks like you didn’t make any calls after this, thus it went undetected by our runtime checks.