Durable messages / transaction handling

Sorry for spamming you with ideas and comments.

I was looking at this piece of code from https://www.cofoundry.org/docs/framework/data-access/transactions:

    public async Task ExecuteAsync(DeletePageCommand command, IExecutionContext executionContext)
    {
        ...
            
        using (var scope = _transactionScopeFactory.Create(_dbContext))
        {
            ...
            scope.QueueCompletionTask(() => OnTransactionComplete(command));

            await scope.CompleteAsync();
        }
    }

    private Task OnTransactionComplete(DeletePageCommand command)
    {
        _pageCache.Clear(command.PageId);

        return _messageAggregator.PublishAsync(new PageDeletedMessage()
        {
            PageId = command.PageId
        });
    }

Now, what happens if the message handler fails? Lets say I’m building an index based on messages like the one above - or deleting related stuff when a page/custom entity/something else is deleted - when the message handler is invoked, the core operation has completed and commited, so if my message handler fails, I loose an update or delete, and the external index gets out of sync.

I would rather have synchronous message handlers like these be handled inside the same transactional boundary as the core operation - in which case both the core operation and the message handler either fails or commits together.

I do realize the issue you have with cashes - you don’t want the cashes to be modified if the transaction commit fails after updating the cache, since caches usually doesn’t share the same transactional behavior (you cannot easily rollback a change to an in-memory cache).

[JornWildt]

Our behavior mimics the behavior of System.Transactions.TransactionScope to be consistent and the behavior should be familiar for those already used to System.Transactions, except with more sensible defaults. System.Transactions.TransactionScope does not auto-complete, so we don’t either.

Sure. I probably shouldn’t have stated it as “I would rather …”. Thanks for clarifying.

but I think what you’re trying to do here is wrap the event messages in an outer transaction?

Yes.

That won’t have an effect because the ambient transaction queues up the completion tasks and will execute them after the transaction completes.

Ah, okay, didn’t think of that.

Override ITransactionScopeManager with your own implemention that changes the behavior. Default implementation is here.

Good point. I could even go a bit further and just add a new interface like ITransactionScopeManager_v2 which itself implements ITransactionScopeManager and adds a new QueueCompletionTask_InTransaction method - while still inherit or build on the original transaction scope manager.

But as stated before, I’m most likely not going to need it - I was just trying to figure out what I was missing here and what behavior I need to rethink. So thanks for taking your time to reply.

[HeyJoel]

Lots to reply to here…

I would rather have the scope auto-completing as long as no exception is thrown

Our behavior mimics the behavior of System.Transactions.TransactionScope to be consistent and the behavior should be familiar for those already used to System.Transactions, except with more sensible defaults. System.Transactions.TransactionScope does not auto-complete, so we don’t either.

I looked at your code here and I should point out that with EF DbContext.SaveChangesAsync() will automatically wrap any code that updates the database in a transaction, so you don’t need to use ITransactionScopeManager here. The same is true with any Cofoundry commands, you don’t need to wrap individual command execution in a transaction, the ITransactionScopeManager is only required when coordinating multiple commands.

In your example above` there’s not much benefit to using transactions, because there’s only one command, but I think what you’re trying to do here is wrap the event messages in an outer transaction? That won’t have an effect because the ambient transaction queues up the completion tasks and will execute them after the transaction completes.

It’s worth clarifying at this point that at the SQLServer level there’s isn’t such a thing as “nested transactions”, as you can only have one transaction per connection. Nested transactions is really just an application-level concept where the ambient transaction is managed by ITransactionScopeManager, in the same way System.Transactions.TransactionScope does.

Anyway, the issue that remains here is that you want completion tasks to be executed in a transaction, and we don’t do that for legitimate reasons, even though in your case they may not apply. To resolve the issue we’d need to provide the option to change the behavior of ITransactionScopeManager to run completion tasks inside the transaction.

Alternatively we could provide a second way of handling messages that does run inside the transaction, but I’m not sure on he design of that, I’d need to think about it.

Some workarounds:

• Override ITransactionScopeManager with your own implemention that changes the behavior. Default implementation is here.
• Fall-back to using System.Transactions.TransactionScope to wrap the code you’re running, that should just wrap everything.