executeRaw could support multiple queries from a single string

Problem

Since prisma migrate is experimental and our team is waiting for #446, I was trying to create a simple migration tool. Our needs aren’t big for now so a simple script would help.

I made a node script that would read .sql files from a scripts folder and run them sequentially:

import { PrismaClient } from '@prisma/client';
import path from 'path';
import fs from 'fs';

const main = async () => {
  const db = new PrismaClient();
  try {
    // Create the migrations table
    await db.executeRaw(
      'CREATE TABLE IF NOT EXISTS `Migrations` (`id` int NOT NULL AUTO_INCREMENT, `executedAt` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP, `migrationName` varchar(255) NOT NULL, PRIMARY KEY (`id`), UNIQUE KEY `Migrations.migrationName` (`migrationName`));'
    );

    // Read all the sql scrips
    const basePath = path.join(__dirname, './scripts');
    const scripts = fs.readdirSync(basePath, { encoding: 'utf8' });
    // For each file
    for (const fileName of scripts) {
      // We have to wrap empty results with try catch. Waiting for https://github.com/prisma/prisma/issues/2867
      try {
        // Check if the migration ran already
        await db.queryRaw`SELECT 1 FROM Migrations WHERE migrationName = '${fileName}' LIMIT 1`;
      } catch {
        // If didn't

        // Get the sql content
        const sql = fs.readFileSync(path.join(basePath, fileName), {
          encoding: 'utf8'
        });

        // Run the sql
        await db.executeRaw(sql);

        // Update the migration table
        await db.executeRaw`INSERT INTO Migrations(migrationName) VALUES ('${fileName}')`;
      }
    }
  } catch (error) {
    console.error(error);
  } finally {
    await db.disconnect();
  }
};

main().catch((e) => console.error(e));

But every time when executing the script, which has multiple queries like CREATE TABLE or ALTER TABLE, executeRaw throws an error:

PrismaClientKnownRequestError: Raw query failed. Code: `1064`. Message: `You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CREATE TABLE IF NOT EXISTS `MyTable` (
  `id` int NOT NULL AUTO_INCREMEN' at line 31`
    at PrismaClientFetcher.request (D:\dev\perimetre\export-connect-backend\node_modules\@prisma\client\src\runtime\getPrismaClient.ts:902:15)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  code: 'P2010',
  meta: {
    code: '1064',
    message: "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CREATE TABLE IF NOT EXISTS `IndustrySector` (\r\n" +
      "  `id` int NOT NULL AUTO_INCREMEN' at line 31"
  }
}

The script is not incorrect as it runs fine with Mysql Workbench. And if I take only one query from the script and make a file with only that. It also runs fine.

Suggested solution

db.executeRaw could support running multiple queries instead of only one

[jeffrafter]

executeRawUnsafe will not execute multiple queries and updateMany is only for updating lots of records for the same value. But if you want to update many records with specific values per record there is not a good pattern for that. It turns out if you are only updating a single field on each record then there is a fun pattern:

UPDATE tableName SET fieldName = CASE WHEN id = 1 THEN 'value1'
WHEN id = 2 THEN 'value2'
WHEN id = 3 THEN 'value3'
WHEN id = 4 THEN 'value4'
ELSE fieldName END;

I wrote a small function that helps build this (and chunk into 10000 records at a time so the query isn’t longer than 4mb):

const updateManyByField = async (model: string, field: string, records: any[]) => {
  const chunks = chunkArray(records, 10000)
  let i = 0
  for (const chunk of chunks) {
    const values: any = chunk.flatMap((record: any) => [record.id, record[field]])
    await prisma.$executeRawUnsafe(
      `UPDATE ${model} SET ${field} = CASE ${chunk.map(() => `WHEN id = ? THEN ?`).join('\n')} ELSE ${field} END;`,
      ...values,
    )
  }
}

To use it I set the field value on a set of records I’ve fetched then

await updateManyByField('TableName', 'fieldName', records)

Edit: this is chunkArray:

export const chunkArray = (array: any, chunkSize: number) => {
  return Array.from({ length: Math.ceil(array.length / chunkSize) }, (_, index) =>
    array.slice(index * chunkSize, (index + 1) * chunkSize),
  )
}

🚨 Note: the ELSE is critical here. This UPDATE covers every row in the table when it runs… but for the non enumerated cases it resolves to fieldname = fieldname which is a noop for the database. That is fine if your absolute set is not wildly large (and will generally be optimized out). However, if you need to restrict it, you should add a WHERE ID IN () statement that limits the UPDATE to the covered set of IDs.

[ivosabev]

Two years later and no progress on something so rudimentary?

In my case the need arises due to another missing API. updateMany doesn’t support multiple updates with different where cases. An example would be:

UPDATE categories SET products = 10 WHERE id = 1;
UPDATE categories SET products = 20 WHERE id = 2;
UPDATE categories SET products = 30 WHERE id = 3;

I cannot do:

db.$queryRawUnsafe(`
  UPDATE categories SET products = 10 WHERE id = 1;
  UPDATE categories SET products = 20 WHERE id = 2;
  UPDATE categories SET products = 30 WHERE id = 3;
`);

neither can I do:

updateMany([
  {data: {products: 10}, where: {id: 1}},
  {data: {products: 20}, where: {id: 2}},
  {data: {products: 30}, where: {id: 3}},  
]);

The working alternative with Promise.all and single update exhausts the connection pool, while sequential run is extremely slow for large number of rows.