Json arrays are not properly stored in underlying postgres jsonb array type

Bug description

The Json[] type seems to be incorrectly stored in the underlying database type jsonb[]. Right now it is stored as the first element of the underlying jsonb[] postgres array as a string instead of properly using the postgres array representation.

Examples:

• [] is stored as {[]} instead of {}
• [{test:"test"}] is stored as {["{\"test\":\"test\"}"} instead of {{"test":"test"}}
• [{ test: "test" }, { test: "test2" }] is stored as {"[{\"test\": \"test\"}, {\"test\": \"test2\"}]"} instead of {{"test": "test"},{"test": "test2"}}

How to reproduce

1. Take the following schema and migrate it using Prisma Migrate

generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

model Test {
  id   String @id @default(cuid())
  test Json[]
}

1. Generate the client and run the following script:

import { PrismaClient } from "@prisma/client";

async function main() {
  const prisma = new PrismaClient();

  const data = await prisma.test.create({
    data: {
      test: {
        set: [{ test: "test" }, { test: "test2" }],
      },
    },
  });
  console.log(data);
  prisma.$disconnect();
}

main();

1. Checkout the stored value in the database

Expected behavior

Prisma should store the Json array properly so that array features provided by postgres can be utilised and existing json arrays can also be used.

Environment & setup

• OS: MacOs 10.15.6
• Database: Postgres 11
• Node.js version: v12.18.2
• Prisma version:

@prisma/cli          : 2.7.0-dev.61
Current platform     : darwin
Query Engine         : query-engine a6a6ecba74cbaeee0756c312b1b0f2502b27e49c (at node_modules/@prisma/cli/query-engine-darwin)
Migration Engine     : migration-engine-cli a6a6ecba74cbaeee0756c312b1b0f2502b27e49c (at node_modules/@prisma/cli/migration-engine-darwin)
Introspection Engine : introspection-core a6a6ecba74cbaeee0756c312b1b0f2502b27e49c (at node_modules/@prisma/cli/introspection-engine-darwin)
Format Binary        : prisma-fmt a6a6ecba74cbaeee0756c312b1b0f2502b27e49c (at node_modules/@prisma/cli/prisma-fmt-darwin)
Studio               : 0.284.0

[dpetrick]

I investigated the issue and the engine exposes the correct API to do the correct things (validated in https://github.com/prisma/prisma-engines/pull/1354/files#diff-75b6d96ece3b54882a54bb29dbe12f1c0c3230024573d1b619ac83fae3c62247R1). However, my assumption is that the client doesn’t use the API correctly, because it runs into a type dilemma.

To use json lists of any form correctly, the list also has to be encoded as a GraphQL list on the transport to the engine:

mutation {
  createOneModel(data: {
    json_list_field: { set: ["Stringified Json 1", "Stringified Json 2"] }
  }) {
    id
  }
}

What the client likely does:

mutation {
  createOneModel(data: {
    json_list_field: { set: "Stringified Json list of Json 1 and Json 2" }
  }) {
    id
  }
}

~Since all list-based APIs support coercion of a single value to a list (so you don’t need to write list syntax all the time), you can provide either [value] or value for a list field with a single value. The engine coerces value to [value] if it encounters a list value but no actual list but a single value.~ Doesn’t seem to be true anymore, my bad.

Alright, what I assume happens is: Because valid Json is just that - valid Json - for the client, it stringifies whatever valid Json it finds and stuffs that into the Json field.

The client needs to either figure out how to distinguish Json[] and Json (that info should be readily available, no?), or we need to make the API super explicit and don’t allow this to happen by design.

[matthewmueller]

That’s a good point for keeping Json[].

I just realized this isn’t really about Migrate. We’ll need to help anyone using Json[] in Client transition.

For example, when we fix this behavior, people that have stored {"{\"test\":\"test\"}"} in their database will suddenly get back ["{\"test\":\"test\"}"] rather than [{"test":"test"}].

Of course we need to make this fix, but it will be a breaking change for Prisma Client who relied on Json[] users and we need to make sure we communicate this.