Add {{path}} as option for messages in rulesets

User story. As a writer of rulesets against a large OA3 spec which is split into many different files that get resolved prior to running Spectral, I want to be able to have the message in the ruleset be able to reference the current {{path}} that the error was triggered from to help guide where the issues were in the non-resolved files (where line numbers are useless).

Is your feature request related to a problem? We use Spectral as a CLI tool in a CI pipeline, and triggers on PR changes to a very hefty spec which gets first resolved, then linting the resolved spec. Currently we have over 300+ routes, and if someone makes a change to a large majority of routes (for example, adding oauth scopes to security), we can write a rule to validate that we actually added something to them all with the following ruleset:

security-zero-or-one-oauth-scope:
    description: Every OAuth2 secured endpoint defines scopes
    given: $..paths.*[?( @property != 'parameters' )].security[0]
    severity: error
    message: 'Errored at {{property}}... Error message is: {{error}}'
    then:
      field: oauth2
      function: length
      functionOptions:
        min: 1

However if the validation fails, we might only see the line number and the property name (in our case, oauth2), of which there might be hundreds to manually sift through in the non-resolved files to find the issue because the line number is meaningless when trying to look at the non-resolved OA3 files(see screenshot below).

Describe the solution you’d like If it’s easy to add in given the context, it would be awesome if we could give the option to dump out the current path which caused the error.

So a message of: message: 'Errored at {{path}}... Error message is: {{error}}' could give a message of: Errored at paths["/v1/errorCodes"].get.security[0].oauth2 ... Error message is: min length is 1

We did experiment with running Spectral against the unresolved OA3 files, but it wasn’t successful with detecting things correctly. I’m not sure what sort of resolving is going on under the hood, but if the solution is us doing something incorrectly with regards to using Spectral against the unresolved spec, that would also be a great solution (since it should identify the offending file, where the line number may be useful depending on how things were resolved).

We are open to any suggestions to help devs easily identify where they’ve broken a rule! Thanks!

Additional context Sample error currently:

Experimenting with using Spectral on an unresolved file produces a bunch of errors which fail to identify path params in this particular file:

[P0lip]

@kylesykes Thanks for the sample rulesets and the portion of your spec. They helped me validate my hunch, I will fix the issue.

EDIT: I’ve got a fix already. Going to open a PR tomorrow or on Tuesday latest.

[kylesykes]

Actually, I think I goofed on my initial guess at the issue. That particular rule doesn’t use a custom function (I have another almost identically named that does however).

I have no idea what’s happening now, or what’s special about this rule versus others that are working fine so here’s some examples.

Here’s a rule with almost the same JSONPath syntax that is appropriately put into the right testsuite (with a custom function, showing that’s not the issue). The regex is just looking for paths that do not end in } (trying to find “collection” type endpoints, ie: /pets versus /pets/{petId})

pagination-endpoints-paginated:
    description: All collection endpoints have the pagination tag
    given: $.paths[?( @property.match('^((?!.*\}$).)*$') )].get
    severity: error
    recommended: true
    then:
        field: tags
        function: pagination_exists

And here’s the one that’s throwing things in the undefined test suite:

pagination-responses-have-x-next-token:
    description: All collection endpoints have the X-Next-Token parameter in responses
    given: $.paths[?( @property.match('^((?!.*\}$).)*$') )].get.responses['200'].headers
    severity: error
    recommended: true
    then:
        - field: X-Next-Token
        function: truthy

Things I can spot that are different is the bottom one traverses a path that gets resolved internally (.responses[‘200’] in the spec is just a $ref to a response object, and i’m checking the headers in that) whereas the top one doesn’t have that issue (the tags are an inline array).

If it’s at all helpful, here’s a barebones skeleton of what I’m working with (the actual spec is huge and way more complicated):

{
    "paths": {
        "/agreements": {
            "get": {
                "description": "Get some Agreements",
                "responses": {
                    "200": {
                        "$ref": "#/components/responses/GetAgreementsOk"
                    },
                    "default": {
                        "$ref": "#/components/responses/Error"
                    }
                },
                "summary": "List agreements",
                "tags": [
                    "agreements",
                    "pagination"
                ]
            }
        }
    },
    "responses": {
        "GetAgreementsOk": {
            "description": "Successful operation",
            "headers": {
                "X-Http-Request-Id": {
                    "$ref": "#/components/headers/X-Http-Request-Id"
                },
                "X-Next-Token": {
                    "$ref": "#/components/headers/X-Next-Token"
                }
            }
        }
    }
}

Fun fact, did you know JSONPath-Plus doesn’t support =~ regexp in JSONPath filters? I didn’t until struggling for 2 days 🤦‍♂