Extended Query: Support Batch Execution

Hi!

The Extended Query protocol enables drivers to submit multiple BIND messages before SYNC. One of the big benefits of using Extended Queries is that you can efficiently send a batch of executions without waiting for round trips for each execution. Pairing that with prepared statements and some simplifications: you send a single PARSE, a bunch of BIND/EXECUTE and a SYNC to find out how things went.

In other words, you’d be able to support something like the following without needing 4 entire round trips. (I’m not recommending this API since it would be a terrible breaking change.)

await client.query({
  name: 'my_query',
  text: 'insert sometable (id, val) values ($1, $2)'
  values: [
    [ 1, 'asdf' ],
    [ 2, 'fdsa' ],
    [ 3, 'qwer' ],
    [ 4, 'uytr' ]
  ]
})

For more information, check out how the JDBC Postgres driver handles a batched execution. There are a few layers to dig through, but this appears to be the core of the code that sends a batch of messages and subsequently sends a single SYNC. NOTE: their driver imposes a limit of 128 records per batch as (apparently) further batching does not improve performance.

[brianc]

yeah a proper “batched query” would be nice. Probably a separate object you pass to client.query or pool.query that was like

const batch = new BatchQuery({
  name: 'optional',
  text: 'INSERT INTO foo (bar) VALUES ($1)',
  values: [
    ['first'],
    ['second']
  ]
})
const result = client.query(batch)
for (const res of result) {
  for (const row of res) {
  }
}

Then the batch query execution could throw if this is false for some validation:

for (const row of config.values) {
  if (!Array.isArray(config.values)) {
    throw new Error('Batch commands require each set of values to be an array. e.g. values: any[][]')
  }
}

something like that. Then it would be explicit.

[abenhamdine]

for pipelining, see the experiment in https://github.com/brianc/node-postgres/pull/2706