Proposal to include monitoring agents exceptions in a new data file

_Issue originally created by user fzipi on date 2019-10-08 20:31:17. Link to original issue: https://github.com/SpiderLabs/owasp-modsecurity-crs/issues/1589._

Type of Issue

New feature.

Description

While working on creating one exception for FP (#1583), the discussion there turned into getting a more holistic approach for monitoring agents.

We can take into account Monit, Nagios, Elasticsearch Metricbeat: anything that does the monitoring exclusively from 127.0.0.1.

For example:

SecRule REMOTE_ADDR "@ipMatch 127.0.01,::1" ...
    SecRule REQUEST_HEADERS:User-Agent "@rx %{tx.monitoring_user_agent_rx}" ...

The list of monitoring agent could be read from a data file.

It should not be enabled by default (which was a common concern), and also we may want to add something related to awareness about SSRF by adding these exceptions.

But this is normally a must for anyone that has production server: they will need monitoring.

[dune73]

Yes, that problem with the tags persists.

If #1886 is adopted, then it’s perfectly OK to use it here of course.

901470 should have the default-UAs empty, should not it? And 901480 should then check for the existence / size of that variable. So unless you define UAs in crs-setup.conf, there is nothing that is disabled.

I like the proposal laid out and I realize, I have been overly defensive above. Sorry. If it’s limited to individual IPs, we should be on the safe ground.

Bonus points for only disabling rules that are commonly triggered by monitoring agents.

Is there sense in limiting this to the HEAD and GET methods?

So all in all, I see this as a useful workaround for a persisting problem.

[fzipi]

I like the approach. We should sort the IDs in the ruleRemoveById. The only thing I see is that we might want to use more tags instead of specific numbers. But in that case, we should have to add additional tags to those rules 🤔