Upcoming changes to the GraphQL Engine, API, and Fields

The purpose of this issue is to surface some work we’re doing on the internals of Keystone Next in preparation for its launch as Keystone 6.

Now that we’ve completed our transition to Prisma and flattened out a lot of the internal complexity related to supporting multiple database adapters, we are re-working how Fields work internally to make them more streamlined and consistent.

Because of how Fields are at the heart of Keystone, this also effectively means re-working (and simplifying, and making more consistent) how we generate Keystone’s GraphQL Schema.

We’re also going to be making some breaking changes to the GraphQL API that Keystone generates - for very good reasons, but this is a big deal, and we’ve held off doing anything like this for years.

So it’s important for us to start broadcasting what’s coming, why, and how to manage the transition. Here’s what’s happening:

1. We’re building a new Field Type system that will dramatically simplify fields

Right now, many of Keystone’s internals work by coincidence, by design. This means that you have to express the filters a field supports; then implement those same filters in the query resolver. You have to express the GraphQL schema a field implements; then write input and output resolvers that implement behaviour to handle those same properties.

Here’s an example of what I mean:

  gqlOutputFields() {
    return [`${this.path}: ${this.getTypeName()}`];
  }
  gqlOutputFieldResolvers() {
    return { [`${this.path}`]: (item: Record<P, any>) => item[this.path] };
  }

That’s a small example; many fields are much more complex, which has a cost in consistency and effort required to ship new features. We also have a lot of magic for dealing with Relationships, or things like the _supportsUnique getter. This is all going away.

In the new system, you’ll only have to express the structure of a Field (similar to how you’d define a type in a type system) and Keystone itself will do the implementation for you.

Here’s a (current) example of what this looks like: the new implementation of a text field

export const text = <TGeneratedListTypes extends BaseGeneratedListTypes>({
  index,
  ...config
}: TextFieldConfig<TGeneratedListTypes> = {}): FieldTypeFunc => () =>
  fieldType({ kind: 'scalar', mode: 'optional', scalar: 'String', index })({
    ...config,
    input: {
      uniqueWhere: index === 'unique' ? { arg: types.arg({ type: types.String }) } : undefined,
      create: { arg: types.arg({ type: types.String }) },
      update: { arg: types.arg({ type: types.String }) },
      orderBy: { arg: types.arg({ type: sortDirectionEnum }) },
    },
    output: types.field({
      type: types.String,
    }),
    views: resolveView('text/views'),
    getAdminMeta() {
      return { displayMode: config.ui?.displayMode ?? 'input' };
    },
  });

If you’re wondering “hang on, where’s the rest?” that’s the point! I didn’t include the type definition for the config, but that’s the whole implementation right there.

It even implements a new feature: a uniqueWhere filter if the index on the field is unique!

2. We’re breaking / improving the public GraphQL Schema that Keystone generates

OK, this is the big one. A consequence of the changes to Fields is that we have to nest everything about a Field - the input and output values, and the filters - under a single key, which is the field’s path.

For output types, this is fine, and we won’t change the schema for any of the fields already in Keystone Next. The input types for most fields will stay the same as well. But we will be changing the filtering syntax and that will need to be updated in front-end apps built against a Keystone Next back-end.

Our hope is that this change isn’t a hard one to keep up with, and we’re not changing the actual functionality. On the up-side, this will make the GraphQL Schemas Keystone generates much smaller and easier to document and use.

Basically, a query previously written like this:

query {
  allUsers(where: { name_eq: 'Jed' }) { id }
}

… will after this update look like this:

query {
  allUsers(where: { name: { eq: 'Jed' } }) { id }
}

Since this is the first breaking change for Keystone’s public GraphQL API in years, we’re going to clean up a couple of other things that are inconsistent or should be better.

For example, at the moment we have this weird meta query for when you want to count items in a list:

query {
  _allUsersMeta(where: { name_startsWith: 'J' }) { count }
}

Confusingly, this accepts all the same input arguments as the main allUsers query, including arguments like skip which don’t make sense for getting a count. After this change, there’ll be a new query specific to getting a count, which only accepts the input types that make sense:

query {
  countUsers(where: { name: { startsWith: 'J' } }) { count }
}

… and we may add others for aggregation, grouping, etc. over time.

We’re genuinely sorry for the work this will create on the front-end of apps built on Keystone Next, but we have to rip the bandaid off and can only promise that the long term benefits of cleaning all this up will be worth it for everyone.

You can expect that, after this change lands, and especially after we close this “next” phase and launch it as Keystone 6, we’re done making breaking changes and the APIs that Keystone generates will be stable for a long time going forward.

Full documentation about how to adapt to these changes will be published when we’ve finalised them, and you’ll also see the diffs in the schemas that are generated when you upgrade.

3. We’re going to be adding a lot more functionality to the built-in fields

Through the Keystone Next transition, we’ve been building on the core GraphQL engine and Field Types that Keystone 5 had. Because it’s been challenging to make deep changes to that system, we’ve focused on rebuilding (all the) other parts of Keystone first; with our new fields system, a lot of improvements we’ve wanted to make are now on the table.

High on our list are improving how null values are handled, and options for controlling that on a per-field level; as well as better handling of required fields, validation, and passing configuration like @map through to the Prisma schema for custom table and column names in the database.

This will also include surfacing better error and validation messages, improving things like showing whether fields are required or invalid in the Admin UI, etc.

As in the text example above, we’ll also be adding support for unique fields to be used as selectors (so you can query or mutate a post by its slug, or a use by their email, etc) and features like multi-column indexes in the database.

---

As mentioned at the top of the issue, getting our Fields and GraphQL engine refactored (followed by some cleanup and settling-in) is the last major thing on our list for releasing Keystone Next as Keystone 6.

More than anything, we’re excited to get this done so we can switch our focus from internal changes we’ve needed to make for ages, to iterating more quickly on user-facing (and developer-facing) improvements to Keystone! And while we don’t take lightly the fact that we’re making breaking changes with this update, many features we’ve been wanting to build will go from “too hard” to “very doable” with this change.

As an aside, custom fields have always been one of the most powerful features of Keystone, but with the current infrastructure they’re a bit too hard to build; we’re hoping that with this new system, they’ll become much more accessible for everyone, and more useful as a result!

[VinayaSathyanarayana]

Will the GraphQL API generated by Keystone 6 be compatible with other open source tools / frameworks like

• Hasura
• RedwoodJs

Assume it will be compatible with JsonLogic

[MurzNN]

Great improvements! Can you also take into consideration my PR https://github.com/keystonejs/keystone/pull/4826 that allow pass the exact idField value when creating items? This is very popular feature when we must control value of creating id field on frontend. Especially when we importing data from some other external database.