[RFC] Introduce package.json "role" field to simplify tooling configuration

Status: Open for comments

Need

We currently have a handful of different kinds of package that can existing within a Backstage project, frontend plugins, backend plugins, the app and backend themselves, library packages, plugin libraries, etc. There’s currently no way for tools to look at a package and reliably say what kind of package it is. There are some patterns we can check for, but nothing really enforces these pattern and that could often lead to incorrect results. We could try things like checking what "build" script is being used, and what suffix the package name has, both they’re both pretty unreliable.

There are also parts of the tooling in the @backstage/cli that you’re supposed to “set up the correct way” for each kind of package, you should use the correct "build" and "start" scripts, and set up .eslintrc.js correctly. Thankfully most of these are created with backstage-cli create, but there are some gaps there, and it makes it difficult to convert packages and update old packages. It also makes it impossible to ship some new features through the CLI without requiring updates of the target packages.

On that topic we’re currently a little bit stuck upgrading Jest because we can’t provide a generated configuration that makes sense. Jest recently switched the default environment away from jsdom, and we would like to follow that path. The trouble is that we’re not able to inspect what kind of package we’re dealing with from the built-in Jest configuration in the CLI, so we can’t reliably select the correct environment.

Proposal

Introduce a well-known "role" field to package.json, which declares the role of the package, for example "backstage-plugin-frontend".

I find “role” to be a good overall term, since it’s not as overloaded as “kind” within Backstage, and “type” is already a reserved field. It’s definitely worth bike-shedding that term a bit though, as it would likely stick. On top of the discussion of the term itself we’d also need to settle on how it would be represented in package.json. For example should it be just "role", or affixed for more context, like "backstage-role"?

The value of the role field would be a simple string enum, for example something like the following set:

"cli"
"web-library"
"common-library"
"node-library"
"backstage-app"
"backstage-backend"
"backstage-plugin-frontend"
"backstage-plugin-frontend-module"
"backstage-plugin-backend"
"backstage-plugin-backend-module"

The package "role" could then be read by various tooling and configuration generators. For example our .eslintrc.js could be shifted to do something like require('@backstage/cli/config/eslint')(require('./package.json')) which then resolves the correct base lint config. The "build" and "start" scripts could be switched out to simply always call backstage-cli build and backstage-cli start respectively.

It would also let us extend our possible tooling quite a bit, it would for example be easier to write repo linters that make sure that there aren’t any forbidden dependencies in place, like a "backstage-app" depending on a "backstage-plugin-backend". It would also open up for more powerful documentation generation and discoverability tools.

Alternatives

One close alternative would be to use a less specific role declaration and perhaps break it down a bit too. For example, a role declaration could look more like this "role": { "platform": "web", plugin: true }. I’m currently in favor of the single identifier though as that lets you encapsulate more semantic meaning in each of the roles, and likely lets us do more powerful things in the future.

A naming alternative that I think is worth calling out is to nest the role inside a top-level "backstage" field, so that it looks something like this in package.json:

{
  "name": "@backstage/plugin-foo",
  "backstage": {
    "role": "plugin-frontend"
  }
}

With that (along with alternatives like “backstage-role”) we can drop the "backstage-" prefix and avoid the discussion of whether it should be "backstage-web-library" or just "web-library".

Risks

There’s a significant risk of lock-in, and I think it’s important that we provide ways to configure the build tooling so that any of the configurations that result from setting a package role are also reachable using flags, specific imports, etc. We should not end up in situations where someone has to use the Backstage CLI with package roles declared, for example one thing to avoid would be any kind of compiler integration that generates code based on the role of imported packages.

One risk of going with a single identifier for the role is that we get an explosion of different roles just so we can cover some matrix. Because of that it would be important to consider what the correct abstraction level is for all of the roles, does both web-library and backend-plugin-frontend-module make sense for example? I think it’s best to err on the side of too many roles to begin with and then perhaps prune them over time, since it’s easy to define aliases but you can’t go the other way around. There would also likely be some utility in @backstage/cli-common to work with roles, making the implementation end of things not get too messy.

[Rugvip]

About to close this one as implementation has begun in #9133. Will leave it open for suggestions/feedback until the end of the week.

The current plan is this:

  "backstage": {
    "role": "plugin-frontend" // e.g.
  }

Along with a few utilities for detecting roles of existing packages, and setting package scripts based on role.

[Rugvip]

One pretty big advantage of adding it to package.json when you’re building tooling is that package registries will return the contents of package.json from their API, but if you want to access files you likely need to be downloading and extracting the archive