Make catalog-model validators easily accessible

Feature Suggestion

A CLI to validate entities in a catalog-info.yaml file, using validators from catalog-model.

Context

Our Backstage instance imports a single catalog-info.yaml describing all our entities. It is generated by fetching and transforming information from multiple sources.

We’d like early feedback on whether the generated catalog is valid, way before importing it in our Backstage instance. Say, when we’re working on the logic that generates this catalog, or when the catalog’s schema changes and our catalog is outdated.

We’ve worked-out a script by importing validators from @backstage/catalog-model to use as a CI check (as shared below). But it took us a while to figure everything out. A readily available CLI (or similar) would have made things easier.

Everyone who’s generating a catalog would benefit from early feedback.

Possible Implementation

A sketch of the script we’ve put together:

import {
  Entity,
  apiEntityV1alpha1Validator,
  componentEntityV1alpha1Validator,
  groupEntityV1alpha1Validator,
  locationEntityV1alpha1Validator,
  templateEntityV1alpha1Validator,
  userEntityV1alpha1Validator,
} from "@backstage/catalog-model";

const VALIDATORS = [
  apiEntityV1alpha1Validator,
  componentEntityV1alpha1Validator,
  groupEntityV1alpha1Validator,
  locationEntityV1alpha1Validator,
  templateEntityV1alpha1Validator,
  userEntityV1alpha1Validator,
];

const validateEntityKind = async (entity: Entity): Promise<boolean> => {
  for (const validator of VALIDATORS) {
      ...
      const result = await validator.check(entity);
      ...
};

const validateEntities = (entities: Entity[]) =>
  Promise.all(entities.map(validateEntityKind)).then(
    (results) => results.filter((r) => r === false).length > 0
  );

// read catalog-info.yaml
validateEntities(entities);

[dtuite]

@vinayvinay we’ve created a validator for our own customers. You can find it here: https://github.com/RoadieHQ/backstage-entity-validator

We haven’t tested it on a multi-entity YAML file yet tbh but it might work for your use case. It can do things like:

1. YAML syntax validation
2. Checking that required Backstage entity properties are in place
3. Checking that annotations are spelled correctly
4. Checking that annotation values are correctly prefixed with “url” where appropriate
5. Checking that URLs are valid

LMK if this is useful. We want to test it out a bit first and see if it’s useful but we’re totally open to contributing parts of it upstream.

[freben]

Yeah if this were to be supported properly by the catalog backend (as a new endpoint, which I think could be useful), we would want to run through the relevant parts of the processing chain.

Doing a dry run sounds like a viable option, but I think it comes with some caveats. For example, if it went all the way to the “try to write” step it might refuse because there would be collisions with the entities already in place. And if you tried to validate the location that was already registered, it might just give up without doing any actual work because it notes that it’s already registered. Etc.

A dedicated validation endpoint does sound useful.