Allow adding refs from pre-processors

Is your feature request related to a problem? Please describe. I would like to implement a custom plugin to conventionally add responses to each operation. For example, I would like to add 400 errors to all operations in my spec without having to specify it each time.

Describe the solution you’d like

Let’s imagine a configuration file with the following entries. It will mean that all operations can return 400 or 422. Potentially, this can be extended with a filter to include only certain operations by convention (e.g. a delete request might not return 400 error).

preprocessors:
    my-custom-pluging/conventional-response:
      '422':
        response:
          $ref: './common/errors/validation-error.response.yaml'
      '400':
        response:
          $ref: './common/errors/bad-request-error.response.yaml'

I’ve tried to implement a PoC for this, but it appears that references are all processed before any plugins are invoked. So I end up with Can't resolve $ref error messages from the bundler.

Describe alternatives you’ve considered The alternative will be to build a rule to enforce this. But this is not the same, because there is no way to auto-fix the issue and it adds some churn on the developer to remember to add all standard errors to operations.

Additional context I understand that this might not go well with the idea of reporting the precise error location in case other validators run on post-processed results. However, maybe it can be solved by restoring the post-processed tree somewhere in the temporary folder and pointing issues there.

[TylerRick]

I’m running into this as well. I added a preprocessor/decator (tried both ways) that automatically adds common responses (400, 401, 403) to all operations so that I don’t have to manually add it to each operation. But bundle fails with a whole bunch of Can't resolve $ref errors.

I tried changing the $ref to both how it is in the source file ('../components/responses/bad_request.yaml') and then tried how it is in the bundle file ('#/components/responses/bad_request') and it doesn’t like either way.

It looked like many of the $refs that it was complaining about were not the ones added by the preprocessor, but seemed to be ones that were already there, which I found amply confusing, and eventually gave up trying to troubleshoot and figure out.

Since I can’t fix it, how about a way to silence it? It said Error was generated by the bundler rule. so I tried to silence it with bundler: off in config as well as with --skip-rule bundler — both with no success — which I find confusing since it says the rule is called bundler, so one might assume it is also skippable/turn-offable by that time.

Fortunately, I did finally find a way to silence the errors and make it bundle it anyway using --max-problems 0 --force.

That is less than ideal, obviously, because I would like to see any problems (other than the 95 seemingly spurious Can't resolve $ref errors, that is).

The output when using --force and '#/components/responses/bad_request'-style references (as long as I have at least 1 operation where the ref was manually added, to trick it into actually including that $refed component in the generated output) does seem to be correct/valid/usable, however, so I’m continuing to use this approach for now.

Needless to say, it would be interested in having a quieter, cleaner (no --force) solution or workaround to this problem/limitation of decorators — even just a cleaner way to silence the errors would be appreciated — Or, too, a completely different, blessed way to add common non-unique responses without duplicating them in every operation…

[konstantin-smolyakov-idt]

Have you considered writing a decorator to do this? I was thinking of a similar thing (common headers) recently (but as a decorator).

I thought about this. However, it means that my referenced schemas are never validated also. If they are not included explicitly anywhere that it means they are unreferenced for the compiler.

Also, I suppose you could have these in your components, and reference the components?

This might be a good workaround, didn’t consider it. However, I prefer to keep all the schemas in separate files. I ended up with writing a validation rule instead as an alternative I mentioned before. It’s not ideal but it helps to keep consistency I try to achieve.

On the other hand, while I understand you want to reduce the duplication, note that you will lose some transparency.

That’s why I’m suggesting to expose post-processed result somehow for the end-user. For example it could stay in some temporary directory not committed into the source control: either in a bundled form or copying the original directory tree. Then the end-user can:

• See the post-processed result with his own eyes (be it a decorator or a pre-processor). I would even say that the decorators are not needed if validators are always run on the end-result.
• Navigate from error messages to this temp location to pinpoint the problem.

I understand that this might be not possible as it’s kind of breaks how the core of this product works. Also it’s a downside as errors are not reported in an original location (not how traditional code works). I guess it’s a common trade-off between configuration vs code problem. E.g. see Terraform vs Pulumi approach (DSL vs programming language). Eventually, engineers tend to make configuration so flexible that it almost becomes the programming language. So maybe we should just describe OpenAPIs using TypeScript functions returning OpenAPI objects 😄

autocompletion in our vs code plugin, etc.

Didn’t know it exists, will check it out 👍