Django - support for transaction.atomic

Hey guys, thanks for a great library.

Context

Just to give you a context, I’d like to use it in Django project to reliably communicate with other microservices and implement Transactional outbox and Message Relay patterns.

Problem

The problem is that currently it’s not possible to save django models and defer a task in one transaction. Example code below:

import procrastinate
from django.db import transaction
from .models import *
from procrastinate.contrib.django import connector_params
from asgiref.sync import sync_to_async

app = procrastinate.App(
    connector=procrastinate.Psycopg2Connector(**connector_params())
)

@app.task
def some_other_task():
     print("Hello Word")

@app.task
@sync_to_async  # Needed to be able to use Django ORM
@transaction.atomic
def send_order_placed_notification():
    some_other_task.defer()  # Task is deferred even when the next line throws IntegrityError (transaction rollback)
    SomeModel.objects.create()

@transaction.atomic
def some_view(request):
    send_order_placed_notification.defer()  # Task is deferred even when the next line throws IntegrityError (transaction rollback)
    Order.objects.create()

Root cause

Psycopg2Connector/AiopgConnector create separate database connection instead of using django.db.connection

[ewjoachim]

I think the easiest way to make it work for Django is to rewrite the procrastinate worker in a synchronous way

Yes, I think so too.

[michaszcz]

I’ve created some draft which we can discuss