[EPIC] Rule Sets

This Epic proposes a concept of Rule Sets. This is a grouping mechanism for rule checks and aims to make Zally implementation more general and allow usage beyond Zalando.

As a Zally user I want to be able to package rule checks in groups And I can better group the rules (by topics, technical level, organization, etc.) So that I can easily implement add my own rule sets to a Zally installation without changing source code of the project

Proposal: Rule Sets

Entities

Check

A check implements one aspect of a Rule. A Rule contains one or multiple checks (1-to-n relationship). Check = (Description, CheckImplementation)

Example: “HTTP method have to correspond with the response code as specified in the standard”

Rule

A rule is an abstract entity, which represents an external guideline which can be automatically checked. Rule = (Id, Title, Type, ExternalLink, Checks) with Type ϵ {MUST, SHOULD, MAY}

Example: “Use standardized HTTP response codes”

Rule Set

A rule set is a bundle or a package of Rules. It should represent a logical grouping of rules (e.g. one company’s guidelines). Rule Sets are developed, tested and published as separate packages. Similar to a plugin mechanism, packages can be added to a Zally installation. Zally server is able to discover them on the startup. RuleSet = (Id, Rules)

Example: “Zalando RESTful API and Event Scheme Guidelines”

Configuration

All discovered Rule Sets are active per default. Administrator of Zally installation can deactivate Rule Sets (per rule-set-id) and specific Rules (rule-set-id-rule-id) via configuration.

Working Packages

1. Ignore Rules by External Ids - #558 ✅
2. Core Interfaces - #559 ✅
3. Package Discovery - #560
4. Statistics for Rule Sets - #561
5. Documentation - #562

Preferably, we should work documentation-driven. 1-4 working packages have to be done sequentially.

[roxspring]

Just wanted to make sure I’m understanding the distinction between the various concepts and trying identify the principles of identifying rules and checks. For example looking at the Zalando Guideline 150:

The RuleSet would be: Zalando Guidelines

The Rule would be the heading from the particular section: MUST Use Specific HTTP Status Codes [150]

The checks would be… one per statement/aspect?:

• You must not invent new HTTP status codes; only use standardized HTTP status codes and consistent with its intended semantics.
• You should use the most specific HTTP status code for your concrete resource request processing status or error situation.
• When using HTTP status codes that are less commonly used and not listed below, you must provide good documentation in the API definition. Note that this suggests that the severity can vary by Check, not simply at the Type level (although maybe the most severe roles up to the Type level??).

Or is the idea that checks would be even more fine grained and broken down by HTTP Method, HTTP Status Code Family, or similar?

[roxspring]

Generally looks good to me.

Where do violations fit in? Based on the existing architecture I’m presuming that each check can produce 0…1 violation, and therefore going forward a rule can produce many violations. (I’m tempted to suggest that each check should be able to produce many violations with each violation referencing a single location within the swagger, as opposed to the current model where a violation can point to multiple occurrences within the swagger definition).

Does “Zally operator can deactivate …” imply that just the end user can deactivate thing? Would be good if as the administrator of my installation I can also deactivate so that end users don’t need to repeat that configuration.