Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add common HTTP headers to tests so logs get cleaner

See original GitHub issue

Describe the bug

Many tests don’t include standard headers, so when performing a test plenty of rules will be fired unnecessarily. We need to fix tests that don’t have Accept or other headers like User-Agent… provided the test was not designed to avoid them.

Steps to reproduce

Run tests and see output.

Expected behaviour

We should match the minimal amount of rules for the test to be successful. This will help with the status page project effort.

Actual behaviour

Just fire some tests for example, for rules 920120, and see:

Message: Warning. Match of "pm AppleWebKit Android" against "REQUEST_HEADERS:User-Agent" required. [file "/etc/modsecurity.d/owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "1405"] [id "920300"] [msg "Request Missing an Accept Header"] [severity "NOTICE"] [ver "OWASP_CRS/3.4.0-dev"] [tag "modsecurity"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "OWASP_CRS"] [tag "capec/1000/210/272"] [tag "PCI/6.5.10"] [tag "paranoia-level/3"]

Which can be fixed and does not affect the test.

Additional context

See status page project.

Your Environment

CRS version (e.g., v3.2.0): v3.4
Paranoia level setting: any
ModSecurity version (e.g., 2.9.3): any
Web Server and version (e.g., apache 2.4.41): any
Operating System and version: any

Issue Analytics

State:
Created 2 years ago
Comments:6 (6 by maintainers)

Top GitHub Comments

1reaction

theseioncommented, Jan 17, 2022

Agreeing on a format for naming would also be nice 😃.

Ideas from the issue chat:

use yq to add standard headers to tests
add a “use standard header” option to tests that configure the test harness to add standard headers

1reaction

RedXanaducommented, Jan 2, 2022

How about test templates? I know we have positivetest.yaml.skeleton but I find that a bit confusing/overwhelming. Maybe it could be tidied up, or maybe just a new “best practice” test template? I.e. a template test with all the appropriate headers pre-filled in (User-Agent, Host, Accept, etc.)

Alternatively, we could document it as part of the upcoming ‘Developers’ section of /docs: “Here is an example of a good test. We add these headers because…”

Top Results From Across the Web

How to Add Custom Headers in ASP.NET Core Web API

Custom Headers allow us to add extra content to our HTTP requests and responses, which we can pass between the client and server....

setting request headers in selenium - webdriver - Stack Overflow

Use another driver/library instead of selenium; Write a browser-specific plugin (or find an existing one) that allows you to add header for request....

Custom request header insertions for non-blocking actions

AWS WAF inserts custom headers into a web request when it finishes inspecting the request. So if you use custom request handling with...

Custom HTTP headers logging

Test execution logs now include custom HTTP header names to enable troubleshooting and debugging of the test plans that require a custom header....

Spring Boot: Setting a unique ID per request | by Daniel López

A common mechanism to provide that ID in the world of web applications is through an http header, so what we'll do is...