access.log and error.log placement

Feature request

Place the access_log and error_log directives inside the main server block instead of http block.

Feature description

Disable access_log at http block: File: /etc/nginx/nginx.conf

# ...
http {
    # ...
    access_log off;
    # ...
}
# ...

Enable per site access_log and error_log at main server block: File: /etc/nginx/sites-available/example.com.conf

server {
    listen                  443 ssl http2;
    listen                  [::]:443 ssl http2;
    server_name             example.com;
    set                     $base /var/www/example.com;
    root                    $base/public;
    # new lines here
    error_log               /var/log/nginx/${server_name}_error.log warn;
    access_log              /var/log/nginx/${server_name}_access.log buffer=512k flush=1m;;
    # ...
}
# ...

Note: the ${server_name} variable must be replaced with the actual server_name set by the user for current website since Nginx does not accept variables in error_log or access_log directives path.

How the feature is useful

Every site has from 3 to 5 server blocks for 301 redirects, this means that access_log will be filled with useless info. Disabling access_log at http level and enabling only for the main server block will ensure cleaner logs. Also, enabling per site access_log will allow a faster i/o (smaller files) and faster spotting of entries (for example using bash tail will retreive only the records belonging to a certain site). The last two arguments buffer=512k flush=1m will massively reduce the i/o frequency by keeping the records in memory up to 512Kb or dumping them to file after 1 minute (in case of not reaching the limit). Enabling per site access_log can also be made optional (for backward compatibility, or for user preference) by enabling the ${server_name}_ part from syntax only when a checkbox is checked. IMHO it should be enabled by default, but any case is better than before.

[SorinGFS]

I there anything wrong or else to do…?

I don’t see anything about personalizing accessLog and errorLog filenames by adding ${server_name} in them, and nothing about accessLog parameters buffer=512k flush=1m like here:

    error_log               /var/log/nginx/${server_name}_error.log warn;
    access_log              /var/log/nginx/${server_name}_access.log buffer=512k flush=1m;

Splitting logs by server_name will allow us to analize logs with ease, log analisers can load those files much faster, especially when we talk about months of logs. Nginx log folder will look like this:

domain.com_access.log
domain.com_error.log
other.net_access.log
other.net_error.log
so on...

The accessLog parameters are also very important. When nginx receives a request it fills the logs in real time, and this can be a problem on servers with high traffic. Using buffer=512k flush=1m parameters instructs nginx to keep the records in memory and dump them into the file after 1 minute, or if bigger than 512kb. This values are tested and can be hardcoded, but for brevity you can also think some inputs to allow the user to set their own values for buffer and flush. IMHO this would be to much, those defaults are ok.

Thank you for your effort.

[SorinGFS]

@MattIPv4

I think having access_log on by default for every domain would be for the best, to maintain backwards compatibility, but having the redirect blocks be off by default seems like a logical breaking change to make.

Totally agree with this.

What’s your view on error_log, should that still be a global option that can be overridden at the domain level, or should it just be at the domain level as well?

As I mentioned earlier I think the error_log setting is useless at nginx.conf http level, and it should be managed exactly as we plan for access_log (inside logging tab). However, there is a difference: error_log has no off option, it can only be configured using log levels. Therefore, IMHO there is no need to have custom configuration for redirect server blocks, existing nginx default would be good enough. And, the setting inside logging tab should strictly refer to main server block, allowing user to customize the path and log level.

Once this plan is realised I have one more new idea for logging, but first let’s get this done.

Sorry about writing here instead the PR, should we continue there?