932100 rule triggered with html decimal entities because of using legacy ISO-8859 charsets family

Description

PROBLEM: HTML decimal entity is blocked if the entity is before the dot sign and space or new line sign. I.e. the value of someargname: Something „The Title”. After space or new line more characters

AFFECTED CHARSETS OF ISO-8859 group: ISO-8859-2, ISO-8859-3, ISO-8859-4, ISO-8859-5, ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-10, ISO-8859-14, ISO-8859-15 are affected because the chars are encoded as decimal html „ and ” entities: someargname=Something+%26%238222%3BThe+Title%26%238221%3B.+After+space+or+new+line+more+characters

NOT AFFECTED CHARSETS OF ISO-8859 group:

1. ISO-8859-1, ISO-8859-9 are not affected because the chars are encoded as unicode hex 84 control and hex 94 cancel chars: someargname=Something+%84The+Title%94.+After+space+or+new+line+more+characters

2. ISO-8859-13 is not affected someargname=Something+%A5The+Title%A1.+After+space+or+new+line+more+characters

3. ISO-8859-16 is not affected someargname=Something+%A5The+Title%B5.+After+space+or+new+line+more+characters

4. ISO-8859-11 could be affected, the first sign encoded as decimal html „ entity, the second one is encoded as unicode hex 94 cancel char. If the dot and space or new line lays after the first sign, the false positive will appear: someargname=Something+%26%238222%3BThe+Title%94.+After+space+or+new+line+more+characters

5. UTF-8 is not affected, the signs are encoded as hex of %E2%80%9E and %E2%80%9D, etc.

POSSIBLE SOLUTIONS:

1. I’ve made lots of regex tests and I’ve made the regex modifying template rule of 932100 with negative lookbehind 3 alternatives (lookbehind requires fixed width) before everything in the regex of the rule:

(?<!&#[0-9]{2})(?<!&#[0-9]{3})(?<!&#[0-9]{4})

to exclude 2, 3 or 4 decimal html entities

i.e.: SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* "@rx (?<!&#[0-9]{2})(?<!&#[0-9]{3})(?<!&#[0-9]{4})(?:;|{||||||&|&&|\n|\r|$(|$((|`|$…

It causes: a) arg value of “Something „The Title”. After space or new line more characters” is passed b) arg value of “;ifconfig” is still blocked c) arg value of “ifconfig;. rm -rf /” is still blocked d) arg value of “ifconfig;. rm -rf /; Something „The Title”. After space or new line more characters” is blocked e) arg value of “;ifconfig Something „The Title”. After space or new line more characters” is blocked f) arg value of “Something „The Title”. After space or new line more characters &&ifconfig” is blocked etc.

1. Removing the first rule in util/regexp-assemble/regexp-932100.txt called (“Special regexp case for the ‘.’ (source) command to prevent FP:”):

'.\s.*

and modify regexp of the rule with assembled string by regexp-cmdline and regexp-assemble

• Why removing the rule is not the best option? What kind of vulnerabilities could be exploited?
• I’ve noticed that if I send “;ifconfig” it is passed, etc. so removing the rule of “source command” dot case seems to be not good solution at all.

THE QUESTION: Is the first solution of negative lookbehind approach acceptable? Could anyone test it more? If yes I’ll pull request the patch to REQUEST-932-APPLICATION-ATTACK-RCE.conf. It seems to be resolver of the issue and works pretty fine.

Audit Logs / Triggered Rule Numbers

2020/09/23 13:38:51 [error] 8439#0: *8406125 [client xxx] ModSecurity: Access denied with code 403 (phase 2). Matched "Operator Rx' with parameter (?:;|{||||||&|&&|\n|\r|$(|$((||\${|<\(|>\(|\(\s*\))\s*(?:{|\s*\(\s*|\w+=(?:[^\s]*|\$.*|\$.*|<.*|>.*|\'.*\'|\".*\")\s+|!\s*|\$)*\s*(?:'|\")*(?:[\?\*\[\]\(\)\-\|+\w'\"\./\\\\]+/)?[\\\\'\"]*(?: (5210 characters omitted)' against variable ARGS:someargname’ (Value: `Something „The Title”. After space or new line more characters’ ) [file “owasp-modsecurity-crs-3.4.0-dev/rules/REQUEST-932-APPLICATION-ATTACK-RCE.conf”] [line “102”] [id “932100”] [rev “”] [msg “Remote Command Execution: Unix Command Injection”] [data “Matched Data: ;. After space or new line more characters found within ARGS:someargname: Something „The Title”. After space or new line more characters”] [severity “2”] [ver “OWASP_CRS/3.3.0”] [maturity “0”] [accuracy “0”] [tag “application-multi”] [tag “language-shell”] [tag “platform-unix”] [tag “attack-rce”] [tag “paranoia-level/1”] [tag “OWASP_CRS”] [tag “capec/1000/152/248/88”] [tag “PCI/6.5.2”] [hostname “xxx”] [uri “/”] [unique_id “537e23be1f5e9e4912c83747da0d0ff5”] [ref “o32,42v956,74”], client: xxx, server: xxx, request: “POST / HTTP/2.0”, host: “xxx”, referrer: “https://xxx/”

Your Environment

• CRS version: 3.4.0-dev
• Paranoia level setting: 1
• ModSecurity version (e.g., 2.9.3): 3.0.3
• Web Server and version: nginx 1.18.0
• Operating System and version: Debian 9

Confirmation

[x] I have removed any personal data (email addresses, IP addresses, passwords, domain names) from any logs posted.

[fzipi]

I think this is part of a new PR 😃

[franbuehler]

So you came to the same conclusion as @piotrpazola with regards to the source command shortcut / alias in the form of the single dot on the command line?

Yes.

I think the question whether there is real danger in this command to be difficult. We need more knowledge in this discussion. I’m going to place a call in the slack channel.

Great idea. Thank you.

Then you come to a different conclusion than @piotrpazola. He thinks ;ifconfig is no longer blocked when you remove the source command, you think it is being blocked just fine.

Yes. I came to a different conclusion.

Hmm. Could it have to do with encoding?

I don’t think so, because ifconfig is another line in this regex source file (https://github.com/coreruleset/coreruleset/blob/v3.4/dev/util/regexp-assemble/regexp-932100.txt#L108) and has nothing to do with the dot. But to be honest, I’m not entirely sure and would be glad to hear from @piotrpazola.

Your plan with shifting the rules seems reasonable to me.

What I can do (hopefully today): Remove the source/dot command in the source file and run @piotrpazola regression tests. Fortunately, he wrote so many regression tests. This is very helpful now.