question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

PostgreSQL 14+ Pipelining Support

See original GitHub issue

PostgreSQL 14 introduced the ability to Pipeline queries.

Are there any plans to support this with asyncpg in any capacity? At the time of writing, I’m looking to optimize large numbers of inserts and updates, and pipelining seems like it could be an interesting solution. I checked the existing issues and didn’t seem to find anything related to this so I thought I’d open a fresh issue/feature request.

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
elpranscommented, Oct 15, 2021

Pipelining is already supported via executemany.

1reaction
elpranscommented, Nov 18, 2021

Pipelining only makes sense if you are sending lots and lots of queries as is the case with bulk insert. For regular queries it makes very little sense in the face of substantial implementation complexity, cancellation issues etc. In your case you can trivially emulate pipelining by querying both tables at the same time like so:

    comments, edits = await conn.fetchrow('''
        select
           (select array_agg(comments.*) from comments where user_id = 5),
           (select array_agg(edits.*) from edits where user_id = 5)
    ''')
Read more comments on GitHub >

github_iconTop Results From Across the Web

Documentation: 15: 34.5. Pipeline Mode - PostgreSQL
While the pipeline API was introduced in PostgreSQL 14, it is a client-side feature which doesn't require special server support and works on...
Read more >
How PostgreSQL Pipeline Mode Works - Percona
Although introduced in PostgreSQL 14, pipeline mode works against any currently supported version of postgres as the enhancement is in the ...
Read more >
Support Postgres 14 Pipelining #1113 - jackc/pgx - GitHub
Apparently Postgres 14 introduces the ability to pipeline queries, similar to batching, but, I think, with asynchronous results possible.
Read more >
Pipeline mode: better performance on slow networks
This article shows how pipeline mode, new with PostgreSQL v14, can improve query performance over slow network connections.
Read more >
Pipeline mode support - psycopg 3.1.5 documentation
The pipeline mode is available on any currently supported PostgreSQL version, but, in order to make use of it, the client must use...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found