Implement more robust JSON Schema parsing for better validation and modularity

• I’d be willing to implement this feature (contributing guide)

Description

I would like Nx to use a more robust implementation of JSON Schema parsing and validation that:

• Allows Draft 7+ schemas (making use of the $schema field), including new keywords and formats
• Resolves external $refs (associated issue)
• Implements the non-spec $data keyword

Motivation

Nx implements its own subset of JSON Schema (more specifically, a subset of Draft 4) and does not include any of the additional formats and keywords provided by Draft 7. This fact makes it more difficult to re-use existing schemas and existing JSON schema tooling (like AJV), so that Nx schemas just sorta have to be their own thing. This reduces modularity and increases manual work by plugin developers, including the Nx team. Further, the Draft 7 spec includes keywords that allow for expressing a wider range of logical scenarios, allowing for the schemas (coupled with the parser/validator) to do a better job of guaranteeing valid input and thus reducing the need to do so in code.

The specific things I would like to be able to do, but cannot with the current implementation, include:

• Use external references in the schemas for the generators/executors of my custom plugins, so that I can re-use shared fields between them instead of copy-pasting and then manually updating. (This is enough of a problem that I am already planning to add this feature to my custom plugin, via a step in the builder that resolves references.)
• In combination with external reference resolution, be able to reference schemas from other drafts and still have them work, so that I can have my Nx schemas directly reference my project data schemas without having to manage a compatibility layer
• Use the proposed (non-spec) $data keyword in combination with Draft 7 $if-$else and other keywords to write schemas that properly validate coupled fields. For example, if I have a generator that can be used to create a library or an app, but some fields are only applicable to one type or the other, being able to capture the full logic of that within a single schema would prevent needing to do it in code.

Suggested Implementation

I suggest replacing Nx’s bespoke JSON Schema parser/validator with AJV.

Benefits

• AJV provides simple mechanisms to add custom keywords and formats, so the custom things that Nx is doing could be transferred there. AJV can also be set to ignore non-spec keywords & formats, so the custom Nx features could also be kept largely as-is while using AJV only for basic validation and parsing
• It has built-in $ref resolution, including cross-file (locally) and web URL references
• It has robust Typescript support, including a robust utility-type JSONSchemaType<T> that makes it a lot easier to write a schema, when starting with a Typescript interface
• It can handle many of the existing JSON Schema specs, in particular all of the latest ones
• It’s already an Nx dependency
• Developers who make extensive use of JSON Schema will likely already be familiar with it
• The Nx team will not have to develop/maintain its own bespoke JSON Schema parser, instead providing plugins for AJV
• The Nx team will not have to have deep expertise in the nuances of JSON Schema Draft specs
• The Nx team will be less likely to have to field support issues related to Nx’s specific implementation (and can point people to AJV)

Risks

• Increases Nx’s dependency on 3rd parties
• Adds development cost to make the switch
• Nx uses its JSON Schemas for other tasks, like generating the Nx Editor UI for generators, and so the Nx team would need to choose between allowing only a subset of JSON Schema period, or only supporting a subset for providing UI generation, or trying to keep up with JSON Schema Draft features to provide full support for Nx add-ons.

Alternate Implementations

In the absence of something as dramatic as switching to ajv, the core value-add could be created through a few changes that don’t require a 3rd party dependency:

• Resolving external $refs takes care of the re-usability problem, which IMO is the most important missing feature in Nx’s implementation.
• Checking for $defs as well as $definitions when resolving $refs would allow Draft 7 Schemas to be directly used by nx, and would decrease the likelihood of conflict with other tools.

[adam-coster]

They have a plugin that adds a new schema keyword (errorMessage) allowing customization of what gets shown.

It’s set up for a wide range of use cases, and lets you specify message details right in the Schema.

On the user side this lets us specify our own error messages (something I’d love to be able to do) and on the Nx side you could fall back on default errorMessage subschemas.

https://ajv.js.org/packages/ajv-errors.html

[FrozenPandaz]

I am open to not maintaining our own schema parser and moving to ajv.

However, one concern I have is that the errors that it provides are not as legible as what we currently have. I feel we provide better error messages. Do you have any insight there?