tx.paranoia_level on v3/master ignored

_Issue originally created by user theMiddleBlue on date 2018-06-15 10:47:26. Link to original issue: https://github.com/SpiderLabs/owasp-modsecurity-crs/issues/1127._

Hi,

I’ve an odd behavior with the last v3/master version (I’ve just pulled it). It seems that it ignores at all the setvar:tx.paranoia_level=1 and it matches many rules in PL2, 3 and 4.

My configuration:

SecAction \
  "id:900000,\
   phase:1,\
   nolog,\
   pass,\
   t:none,\
   setvar:tx.paranoia_level=1"

And on audit log I’ve many of these (with my previous version of v3/master it wasn’t happen):

{
  "details": {
    "accuracy": "0",
    "rev": "",
    "ver": "",
    "reference": "v1839,1",
    "lineNumber": "18",
    "match": "Matched \"Operator `Eq' with parameter `0' against variable `MULTIPART_UNMATCHED_BOUNDARY' (Value: `2' )",
    "tags": [],
    "ruleId": "200004",
    "maturity": "0",
    "data": "",
    "severity": "0",
    "file": "/usr/local/openresty/nginx/conf/modsecurity_config/http-www.acselspa.it/http-www.acselspa.it"
  },
  "message": "Multipart parser detected a possible unmatched boundary."
},
{
  "details": {
    "accuracy": "9",
    "rev": "2",
    "ver": "OWASP_CRS/3.0.0",
    "reference": "o40,1o41,1o87,1o88,1o89,1o90,1o110,1o111,1o152,1o153,1o201,1o202,1o203,1o204,1o215,1o216,1o257,1o258,1o305,1o306,1o307,1o308,1o313,1o314,1o355,1o356,1o412,1o413,1o414,1o415,1o428,1o429,1o472,1o473,1v1839,474t:urlDecodeUni",
    "lineNumber": "1287",
    "match": "Matched \"Operator `ValidadeByteRange' with parameter `32-36,38-126' against variable `REQUEST_BODY' (Value: `------WebKitFormBoundaryWWc2IeyPuspjWwJ5\\x0d\\x0aContent-Disposition: form-data; name=\"action\"\\x0d\\x0 (476 characters omitted)' )",
    "tags": [
      "application-multi",
      "language-multi",
      "platform-multi",
      "attack-protocol",
      "OWASP_CRS/PROTOCOL_VIOLATION/EVASION",
      "paranoia-level/3"
    ],
    "ruleId": "920272",
    "maturity": "9",
    "data": "",
    "severity": "2",
    "file": "/usr/local/openresty/nginx/conf/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"
  },
  "message": "Invalid character in request (outside of printable chars below ascii 127)"
},
{
  "details": {
    "accuracy": "9",
    "rev": "2",
    "ver": "OWASP_CRS/3.0.0",
    "reference": "o40,1o41,1o62,1o72,1o73,1o79,1o86,1o87,1o88,1o89,1o90,1o110,1o111,1o152,1o153,1o174,1o184,1o185,1o191,1o200,1o201,1o202,1o203,1o204,1o215,1o216,1o257,1o258,1o279,1o289,1o290,1o296,1o304,1o305,1o306,1o307,1o308,1o313,1o314,1o355,1o356,1o377,1o387,1o388,1o394,1o411,1o412,1o413,1o414,1o415,1o428,1o429,1o472,1o473,1v1839,474t:urlDecodeUni",
    "lineNumber": "1348",
    "match": "Matched \"Operator `ValidadeByteRange' with parameter `38,44-46,48-58,61,65-90,95,97-122' against variable `REQUEST_BODY' (Value: `------WebKitFormBoundaryWWc2IeyPuspjWwJ5\\x0d\\x0aContent-Disposition: form-data; name=\"action\"\\x0d\\x0 (476 characters omitted)' )",
    "tags": [
      "application-multi",
      "language-multi",
      "platform-multi",
      "attack-protocol",
      "OWASP_CRS/PROTOCOL_VIOLATION/EVASION",
      "paranoia-level/4"
    ],
    "ruleId": "920273",
    "maturity": "9",
    "data": "",
    "severity": "2",
    "file": "/usr/local/openresty/nginx/conf/owasp-modsecurity-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"
  },
  "message": "Invalid character in request (outside of very strict set)"
},
{
  "details": {
    "accuracy": "8",
    "rev": "2",
    "ver": "OWASP_CRS/3.0.0",
    "reference": "o2,8o2,8v1931,19t:urlDecodeUni",
    "lineNumber": "1251",
    "match": "Matched \"Operator `Rx' with parameter `((?:[\\~\\!\\@\\#\\$\\%\\^\\&\\*\\(\\)\\-\\+\\=\\{\\}\\[\\]\\|\\:\\;\\\"\\'\\´\\’\\‘\\`\\<\\>][^\\~\\!\\@\\#\\$\\%\\^\\&\\*\\(\\)\\-\\+\\=\\{\\}\\[\\]\\|\\:\\;\\\"\\'\\´\\’\\‘\\`\\<\\>]*?){2})' against variable `ARGS:action' (Value: `wp-remove-post-lock' )",
    "tags": [
      "application-multi",
      "language-multi",
      "platform-multi",
      "attack-sqli",
      "OWASP_CRS/WEB_ATTACK/SQL_INJECTION",
      "WASCTC/WASC-19",
      "OWASP_TOP_10/A1",
      "OWASP_AppSensor/CIE1",
      "PCI/6.5.2",
      "paranoia-level/4"
    ],
    "ruleId": "942432",
    "maturity": "9",
    "data": "Matched Data: -remove- found within ARGS:action: wp-remove-post-lock",
    "severity": "4",
    "file": "/usr/local/openresty/nginx/conf/owasp-modsecurity-crs/rules/REQUEST-942-APPLICATION-ATTACK-SQLI.conf"
  },
  "message": "Restricted SQL Character Anomaly Detection (args): # of special characters exceeded (2)"
}

Any idea? how can I do for pull the previous version of v3/master?

Thanks!

[CRS-migration-bot]

User victorhora commented on date 2018-06-20 23:10:17:

Thanks for the detailed reporting guys.

This is being handled at https://github.com/SpiderLabs/ModSecurity/issues/1808. PR #1810 is up for evaluation to fix this issue. My tests went well and the buildbots are going fine for now. Would appreciate if you folks could test and report as well.

Thanks!

[CRS-migration-bot]

User michaelgranzow-avi commented on date 2018-06-20 13:10:58:

I think this bug has been introduced by https://github.com/SpiderLabs/ModSecurity/commit/892beb53603a44a3b7cdcbfe83eff9e8c51c9307

Here, the lookup was changed from using std::multimap::equal_range with custom hash and equal functions (both case-insensitive) to std::string::compare, which is case-sensitive:

-        auto range = this->equal_range(var);
-        for (auto it = range.first; it != range.second; ++it) {
-            l->insert(l->begin(), new VariableValue(&m_name, &var, &it->second));
+        for (auto &a : *this) {
+            if (a.first.compare(0, var.size(), var) == 0) {
+                l->insert(l->begin(), new VariableValue(&m_name, &var, &a.second));
+            }

As a side-note, this change also seems to have turned a hash lookup into a linear search, so it may well have performance impact.