Rethink dependency auto-install during Prisma Client's generation

I’m specifically talking about this part of the Prisma Client spec: https://github.com/prisma/specs/tree/master/cli#install-prismaclient

I’m told that the reasoning behind introducing it was to smoothen the first user experience, but I urge you to rethink our solution.

Here are the problems I see currently:

1. This smells like an anti-pattern. prisma2 generate is touching and modifying files outside its own “scope”. This feels like an abuse of the lack of a permissions system in Node. It is not prisma2’s responsibility to manage dependencies. In less technical terms, it is too “magical”.

2. We’re making assumptions about the end user’s setup. They might be working with yarn/npm workspaces, might have other symlinks in their node_modules that will be disturbed by a yarn install command, or they might not even be using npm or yarn to manage dependencies at all (think pnpm, lerna or yarn2) The point is, we don’t have enough information to figure all this out, and we shouldn’t need to. As an example: In Studio’s tests, I use the SDK to generate the Client (in other words, the programmatic API of Prisma Client).

3. This is not CI / test friendly. If I run prisma2 generate during a test, it will have a side-effect of modifying my package.json after it has run. This might not be as significant, but it highlights the fact that our current solution does not take a lot of use cases into account.

Here are some solutions I propose. Please correct me if I’m missing some context:

1. When I run prisma2 generate in a project where we’re able to find a schema.prisma file but the generator cannot be resolved, error out. If the error clearly tells me that I need to install a separate package to use the generator I have defined in my prisma.schema, I do not see it as a friction point in our “Getting Started” experience.

(Moving 2. to prisma/specs#452 because I don’t think it belongs here on second thought) 2. Instead of going back to the old way of running prisma2 generate in postinstall hooks, we could show a warning in development mode about Prisma Client being “out of sync” when it detects a schema.prisma change. This could happen during the Client’s first connect() call, which happens automatically. (AFAICT, a nasty side-effect of a failed postinstall script in a package is that the user cannot even npm i without getting rid of the faulty package, which is why I’m against a postinstall script in the first place.) We might also be able to look at tools like Prettier and run prisma2 generate whenever schema.prisma changes as part of our VS Code plugin. This obviously won’t work for people on Sublime etc., though.

[zachasme]

This appears to have been re-introduced: https://github.com/prisma/prisma/blob/b337d3c2bf3c4278da45912b8ca5cdc9a64ab894/src/packages/sdk/src/predefinedGeneratorResolvers.ts#L78

While it certainly is helpful in simple setups, I still think the points made by @madebysid are valid, specifically point 2

We’re making assumptions about the end user’s setup.

I’ve have two problems with this behavior so far:

• In a previous project, when trying to keep general dev dependencies like @prisma/cli in my root workspace and only have @prisma/client in my service workspaces, this kept putting @prisma/cli into my package.jsons. This was slightly annoying since I was trying to only keep runtime dependencies in the last stage of my docker build.
• In my current project I’m trying to use yarn 2 pnp. I’m generating the js client into a folder that I use as a workspace, which I then link using the workspace: range. In this scenario I don’t need to @prisma/client wrapper package.

Related https://github.com/prisma/prisma/issues/1439#issuecomment-698398513

[chrisui]

Just to try and add some additional context we have unfortunately found prisma within a monorepo context somewhat untenable. Slightly exagerating the scope of the original issue here but it goes to highlight how decisions made for ease of first usage don’t scale well. Using Nx to manage tasks within a basic npm workspaces repo, we have found the following pain points:

1. generating into node_modules by default causes tools such as Nx to freak out as typically node_modules is ignored for hashing states. Luckily we can give an output for the client but…
2. now @prisma/client is empty and throws an exception when something tries to resolve it. luckily we just change our imports, but…
3. now when bundling (ie. esbuild) we cannot easily configure “externals” which meant bundles would properly resolve before… ok but we can just bundle the client and not make it external… yes, but…
4. now paths are all messed up. in the generated prisma client code __dirname is used excessively and suddenly we have to copy schema files, binaries and runtimes all over the place to keep a bundle portable…
5. oh and now we see random npm warnings when running prisma generate which confused many of us (and is actually how I ended up on this issue in the first place trying to figure out what was happening)
6. I’m yet to try and figure out what happens if we wanted two schemas in this repo and npm hoists @prisma/client to the root (I think this is a key point and the fatal flaw with the concept of @prisma/client being a npm package that is actually something generated locally - ie. npm specifically de-dupes packages because they should be considered immutable, so if I have service-a and service-b both wanting to generate a client, we still end up with a random @prisma/client in the top level node_modules)
7. The copying of binaries into the client causes huge bloat with caching strategies such as Nx. We should just be able to re-use binaries installed into a binaries/engines package and not installed and copied into a generated runtime client.

To throw a few other pains in which really become noticeable in a monorepo setting:

1. (a as Prisma.Decimal) instanceof (b as Decimal)
2. client/index.js grows linearly and without any use of es modules (or at least #__pure comments) reduces the ability to effectively optimise bundles (particularly a serverless function setting)

This article about creating an nx generator for prisma really shows the length needed to go to in order to get prisma clients functioning predictably in a monorepo - and the result of “there is one magic package that provides a gateway into other individual schemas” defeats the purpose of being able to make a granular monorepo with small schemas/clients.

Suggestions:

1. Dont generate into node_modules - install re-usable binaries into it
2. Do away with @prisma/client - let the developer generate where they want and import however they want - it’s fundamentally broken (I never say this lightly, I’ve struggled with it for months to finally understand what the issue is) to have an installable package that is then expected to be mutated by the consumer - a package forces a to-one dependency whereas the reality is we always want to-many dependency on N clients
3. Dont autoinstall things on “generate”. Auto install when we want to install things. At least prompt for explicit action and allow the developer to resolve the issue which would trigger the need for an autofix.
4. Dont copy/rely on binaries relative to a generated client - have a constraint that means binaries will only be found in a) a single package (not a user-generated package) and/or b) a custom installation path determined by the user (already supported)
5. Be more portable when it comes to paths and referencing source schemas - probably compile schema into the generated client rather than make assumptions on where the generated client will be bundled.

Sorry this turned into a bit of a dump but hopefully it helps somebody on the prisma team understand the problems in a more fundamental way. I’d be very happy to hop into slack and discuss further - even contribute if there’s a clear path!