Huge inserts fail

I tried to paste together code from different sources and boil down the example as much as possible so I apologize for the messy code: https://gist.github.com/Rauno56/c3369b58111ffbe17ff63d64289e4aa6 On my setup this fails with

bind message supplies 26784 parameters, but prepared statement "" requires 420000

even though parameter count and placeholder count match(see from the outputs).

If you were to change N smaller or chunk up the request it does fine. Seems like the input of parameters is cut short at some point.

[Rauno56]

Could not use UNNEST in my case as the db version was tiny bit short of supporting it. Precompiled using pg-promise’s helpers. I think this limit of parameters should at least be documented and alternatives provided here, in pg module too.

Thank you very much @charmander @vitaly-t

[ugultopu]

Arrays worked like a charm, thanks @charmander ! Unrelated to this issue, I have one question that maybe you can help: I’m trying to implement an “optional upsert”. That is:

• If the record does not exist, insert.
• If the record exists, for certain columns, check the parameters to understand if they should be updated. If yes, update them. Otherwise, leave them as-is.

Here’s what I came up with so far:

INSERT INTO table_name (
  "col1",
  "col2",
  "col3",
  "col4",
  "col5",
  "col6",
  "col7"
)
SELECT DISTINCT
  a."col1",
  a."col2",
  a."col3",
  a."col4",
  a."col5",
  a."col6",
  a."col7"
FROM UNNEST (
  $1::uuid[],
  $2::uuid[],
  $3::numeric[],
  $4::numeric[],
  $5::boolean[],
  $6::boolean[],
  $7::timestamptz[],
  $8::boolean[],
  $9::boolean[],
  $10::boolean[]
) WITH ORDINALITY AS a(
  "col1",
  "col2",
  "col3",
  "col4",
  "col5",
  "col6",
  "col7",
  "shouldUpdateCol3",
  "shouldUpdateCol4",
  "shouldUpdateCol5",
  "ordinality"
)
ON CONFLICT
  ("col1", "col2")
DO UPDATE
SET
  "col3" = CASE WHEN EXCLUDED."shouldUpdateCol3" = TRUE THEN EXCLUDED."col3" ELSE table_name."col3" END,
  "col4" = CASE WHEN EXCLUDED."shouldUpdateCol4" = TRUE THEN EXCLUDED."col4" ELSE table_name."col4" END,
  "col5" = CASE WHEN EXCLUDED."shouldUpdateCol5" = TRUE THEN EXCLUDED."col5" ELSE table_name."col5" END,
  "col7" = EXCLUDED."col7";

The problem is, since shouldUpdateCol3, shouldUpdateCol4 and shouldUpdateCol5 are not selected in the SELECT FROM UNNEST above, this does not work.

However, if I add them to the SELECT FROM UNNEST, then I get INSERT has more expressions than target columns error.

Do you know how I can accomplish this?