implement pipelining mode allowed by libpq 14

libpq shipped with Postgres 14 now allows using pipelining mode https://www.postgresql.org/docs/14/libpq-pipeline-mode.html

as stated by these docs

While the pipeline API was introduced in PostgreSQL 14, it is a client-side feature which doesn’t require special server support and works on any server that supports the v3 extended query protocol.

So I don’t know exactly when v3 extended query protocol has appeared (at least 2010 I guess), but it’s there since a lot of time, so probably all supported version of pg server (9.6 and above) are able to support pipelining mode.

related commit in posgres repository is there https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=acb7e4eb6b

For the record, a previous PR has been proposed at https://github.com/brianc/node-postgres/pull/662 but it was before before libpq supports it so the PR has been closed because it would have created a discrepancy between native bindings (which use libpq) and the js client modified by the PR.

IMHO this mode should be opt-in, at least for current major version :

• because it’s not so clear (to me at least) how some workflows are handled (what if you send a DDL query adding a column and then immediatly a DML query using the new column ?)
• to avoid subtle breaking changes in users usage (especially for old version of pg)
• for memory consumption reason (from the pg docs : Pipeline mode also generally consumes more memory on both the client and server)

[abenhamdine]

But when using a pool, pipeline would get distributed among all the clients

Indeed, but it will be the case with or without pipelining, thus why would it be an issue ? Or perhaps I miss something in your explanation ?

EDIT : oh sorry, it looks like I missed the important part of your post :

so other queries won’t be scheduled in between

Yes it’s part of my thoughts (also with mixing DML/DDL queries)
Perhaps indeed it should be allowed to optin (or optout ?) at the pool and/or client level

[elchinoo]

Saving network tripes can provide significant gain perfs, see comments and benckmark https://github.com/brianc/node-postgres/pull/2706 (nodejs implementation but the saving of network trips would be the same I assume)

I have no doubt that when in high latency network the gains could be significant but was skeptical on how much gain we could get otherwise. I’m still concerned of the complexity it may add to some client implementations but after reading the points (and benchmark) in the cases you referred I’m starting to rethink it. Thanks for sharing!