Rule 920100 does not handle CONNECT requests correctly

Description

Issue originally reported on StackOverflow, here.

Rule 920100 is designed to “Validate request line against the format specified in the HTTP RFC”. It includes support for the HTTP CONNECT request method. Unfortunately, the rule only recognises a subset of valid CONNECT requests.

The rule will allow a CONNECT request that uses an IPv4 address on the request line.

The rule will not allow a CONNECT request that uses a domain on the request line. This causes a false positive.

RFC 7231 gives an example of using a domain name in this way, so this behaviour is to be expected and should be allowed and accounted for:

CONNECT server.example.com:80 HTTP/1.1

Further research and testing required, and then rule 920100 probably needs to be expanded to account for this case.

Additional question: ☑ How about IPv6? Is there an RFC allowing for that?

Answer: Possibly, but I want to avoid getting stuck figuring out the semantics of RFCs. So, I’m choosing to follow RFC 2817, which talks about CONNECT request lines using a “host name and port number destination… separated by a colon: CONNECT server.example.com:80 HTTP/1.1”.

Additional question: ☑ Is a t:lowercase transformation more efficient than a case insensitive regular expression? Could that improve the performance of this rule?

Answer: When tested without JIT enabled, it’s 54.1 vs 52.1 μs in phase 2 for using (?i:) vs t:lowercase.

Tested again using the CRS Docker images, and concluded that switching to t:lowercase does not yield an improvement. It seems to be quite situational (e.g. one large payload vs many small payloads).

Additional question: ☑ Can the regular expression in rule 920100 be disassembled?

Answer: Yes! This is now done.

Useful command: Use cURL to construct a valid CONNECT request for testing:

curl -v -o/dev/null -p -x localhost:80 http://test.com

Audit Logs / Triggered Rule Numbers

[Tue Oct 05 20:18:38.418643 2021]
        [:error]
        [client 192.168.85.1]
      • ModSecurity: Warning. Match of "rx ^(?i:(?:[a-z]{3,10}\s+(?:\w{3,7}?://[\w\-\./]*(?::\d+)?)?/[^?#]*(?:\?[^#\s]*)?(?:#[\S]*)?|connect (?:\d{1,3}\.){3}\d{1,3}\.?(?::\d+)?|options \*)\s+[\w\./]+|get /[^?#]*(?:\?[^#\s]*)?(?:#[\S]*)?)$" against "REQUEST_LINE" required.
        [file "/opt/httpd-waf/modsecurity.d/rules_3-3-0/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"]
      • [id "920100"]
        [msg "Invalid HTTP Request Line"]
        [data "CONNECT test.com:80 HTTP/1.1"]
        [severity "WARNING"]
        [ver "OWASP_CRS/3.3.0"]
        [tag "paranoia-level/1"]
        [hostname "test.com"]

Your Environment

• CRS version: Tested against v3.4/dev, v3.3.2, v3.3.0
• Paranoia level setting: PL1
• ModSecurity version: Any
• Web Server and version: Any
• Operating System and version: Any

Confirmation

[X] I have removed any personal data (email addresses, IP addresses, passwords, domain names) from any logs posted.

[thatSecOpsGuy]

Hi,

original stackoverflow issue reporter here. As hinted by @RedXanadu on Stackoverflow, using CONNECT might lead to other false positives.

I do have one or two rules besides this one causing a warning with CONNECT. Should I report that here or open a separate issue for investigation?

And @dune73: indeed, I did not expect such a great response. Again thanks @RedXanadu .

[thatSecOpsGuy]

While testing, I’ve noticed that valid CONNECT requests trigger rule 920300: “Request Missing an Accept Header” at PL 3.

After a bit of research, it looks like CONNECT requests don’t include Accept headers, and this is expected and acceptable.

Interesting: in the field I do not have any issues with 920300 and CONNECT requests. I do however have issues with 920280. I will open another issue regarding this.

Update: I just noticed you are using PL3. That is likely why you are seeing issues with 920300 while I am not. Update2: After some investigation, the issue with 920280 is not specific to the CONNECT method but rather seems to be a general issue with that rule and http/1.0. Will investigate further…