Integrating rule check into Travis CI

_Issue originally created by user airween on date 2019-10-16 20:07:51. Link to original issue: https://github.com/SpiderLabs/owasp-modsecurity-crs/issues/1599._

Extending the continuous integration tests

We discussed about the more checks before, and now csanders-git asked again, I would like to start to design the implementation.

Description

The main goal is to extend the Travis CI with some more tests, which did not exist before, and helps to keep the well formed text structure of the whole rule set. Ideas have come before, now I try to collect and organize them, and please help me to make the finally plan.

To make the new checks, I’ld like to use msc_pyparser to build the AST, write the abstract structure, and re-generate the rule set. Between these two steps, I’ld like to run the test cases, so the main line:

1. generate structures
2. run checks
3. write the parsed structure

Let’s see the function of each steps!

1. For a correct work it’s necessary that the rule sets must be syntactically (and of course lexically) correct. Now the secrule_parser from fzipi is a good tool, but any case sensitive differences could break the check. I don’t know that could be changed or not, but I think we can leave it in the CI.

2. In this step, I got these ideas from you:

• case-sensitive check for configuration directives, variables, operators, and actions
• order check for actions - see the related wiki and doc
• occurrence of mandatory tags, actions:
  • ver
  • paranoia-level
• collect referred but non-existent tags (ctl:ruleRemoveByTag=TAG but TAG not exists)
• and many other ideas what you have

1. Although this may not seems important, but the writer of parser class is very strict. I mean it uses predefined (hard-coded) indentations, leading and trailing spaces (eg. before the trailing \ at the end of lines, and so on…), so with help of the re-generated rule set, we can use a simple diff to detect the hidden (or missing) spaces, tabs and other annoying characters.

There is one more type of checks, which isn’t important and doesn’t have any additional worth. We can check many other things, eg. uniqueness of rule ID’s, mandatory and optional arguments at variables, operators and actions - but (and that’s why I wrote that this isn’t a plus worth) this kind of errors are discoverable, because the Apache inside the Travis loads the rules, and it catches these errors.

I hope that this will helps you to get started the ideas.

[CRS-migration-bot]

User fzipi commented on date 2020-03-04 00:23:31:

We need this one open for a little bit longer.

[github-actions[bot]]

This issue has been open 120 days with no activity. Remove the stale label or comment, or this will be closed in 14 days