cannot use Connection.transaction() in a manually started transaction

• asyncpg version: 0.26.0
• PostgreSQL version: 13.7
• Do you use a PostgreSQL SaaS? If so, which? Can you reproduce the issue with a local PostgreSQL install?: -
• Python version: 3.10.4
• Platform: -
• Do you use pgbouncer?: No
• Did you install asyncpg with pip?: Yes
• If you built asyncpg locally, which version of Cython did you use?: -
• Can the issue be reproduced under both asyncio and uvloop?: Yes (asyncio and uvloop used with FastAPI)

I’m seeing sqlalchemy.exc.InterfaceError: (sqlalchemy.dialects.postgresql.asyncpg.InterfaceError) <class 'asyncpg.exceptions._base.InterfaceError'>: cannot use Connection.transaction() in a manually started transaction error, but with some inconsistency.

I have a backend server running(built with FastAPI), and one of the endpoint does the following:

• SELECT some data from the database
• do some operations on the data
• INSERT some data into the database

It is after the INSERT statement that i’m getting the error. But the funny thing is, about 1/5 times I don’t get that error and the endpoint finishes successfully. I searched about the error but found little success. I dont’ have code that ‘explicitly’ start any transactions. By looking at the logs, I do see ‘implicit’ begin statements(created by asyncpg I assume?). Any idea how I could fix this error? First I thought maybe me not committing after the SELECT could be the issue, but now I’m completely lost after some of them completed successfully.

[adriangb]

overall we do see a lot of people using FastAPI having various integration issues though I don’t know that there’s any intrinsic issue in any of the components used

Indeed I’ve seen issues like this reported for FastAPI quite a bit, but it’s really hard to pinpoint where the issue originates from given all of the wrapping and concurrency stuff going on. For example, this code may or may not split a context manager across different threads depending on server load (how busy the thread pool is). If there’s context variables going around, things get even trickier. I’m happy to help debug things on the FastAPI/Starlette side but it would be really helpful to have more reproducible examples (even if the example consists of starting a server and loading it with wrk or something).

[kglee0]

Currently my team is in a different phase of development so I didn’t have time to check on the connections lately, but seems like next time I encounter this issue I would look into FastAPI communities. Thanks for the help!