DurableTask.AzureStorage: CreateTimer fails to complete if called after a continue as new

On DurableTask.AzureStorage, the following orchestration hangs; stepping through the debugger reveals that the timer does not fire.

        public override async Task<string> RunTask(OrchestrationContext context, int input)
        {
            if (input == 0)
            {
                context.ContinueAsNew(1);

                return "continue as new";
            }
            else if (input == 1)
            {
                await context.CreateTimer(context.CurrentUtcDateTime, 0);

                return "OK";
            }

[sebastianburckhardt]

I think I understand the cause for the bug, it is indeed related to the fast ContinueAsNew introduced in #251.

What happens is that the loop inside OnProcessWorkItemAsync goes into the second execution of the instance, which has a different execution id, but the execution id in OrchestrationSession is not updated. When the timer fires, the OrchestrationSessionManager executes AddMessageToPendingOrchestration but because the execution id does not match, the message is put back into the queue (with the expectation that the session will disappear and then a new instance would be created by OrchestrationSessionManager.GetNextSessionAsync to handle the message batch). However, this never happens with extended Sessions and fast continueAsNew enabled, so the message is never actually delivered.

I am working on a fix that can update the execution id in OrchestrationSession and deliver the messages to it.

[cgillum]

I think this might be caused by the fast-ContinueAsNew change in DurableTask.Core 2.1.0. I believe it may also be causing issues with calling activity functions immediately after ContinueAsNew (some discussion of this here: https://github.com/Azure/azure-functions-durable-extension/issues/529).