Namespace MDC Fields with either labels or tags.

I noticed when I was working with the ECS Java loggers that MDC and/or Metadata fields weren’t name spaced.

I can see one of three scenario’s:

1. Make the default namespace be labels per [https://www.elastic.co/guide/en/ecs/current/ecs-base.html](Base Fields)
2. Make the default namespace be tags per [https://www.elastic.co/guide/en/ecs/current/ecs-base.html](Base Fields)
3. Allow for a customized namespace via a configuration parameter.

Here’s an example configuration:

<configuration>
    <appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="co.elastic.logging.logback.EcsEncoder">
            <serviceName>my-application</serviceName>
            <serviceNodeName>my-application-cluster-node</serviceNodeName>
            <additionalField>
                <key>environment</key>
                <value>dev</value>
            </additionalField>
        </encoder>
    </appender>

    <logger name="co.elastic" level="DEBUG"/>

    <root level="WARN">
        <appender-ref ref="Console"/>
    </root>
</configuration>

and a resulting output:

{
"@timestamp": "2021-08-31T15:57:11.355Z",
"log.level": "DEBUG",
"message": "Example with the default Meta Data fields.",
"ecs.version": "1.2.0",
"service.name": "my-application",
"service.node.name": "my-application-cluster-node",
"event.dataset": "my-application.log",
"process.thread.name": "main",
"log.logger": "co.elastic.DemoApplication",
"environment": "dev"
}

[rdifrango]

I think MDC serves a difference use-case than general structured logging. Usually I see MDC used to just dump the current “context” of a transaction. This includes the transaction-id, the logged in user… but really, this can be anything attached to the current transaction context. In the Go programming language, there is usually a context object which is really just a set of key-value-pairs that can contain everything. Spring uses a TransactionContext for similar use cases. This is what I often see being dumped into the MDC.

I was going to make a similar comment that MDC is often used contextual fields that span across all methods within a given flow. For example, if you use distributed tracing that’s something that needs to be present in all log records and the only way to do so is via MDC unless you want to pass that as a direct parameter to all methods.

[ChristianCiach]

We, too, are sad to see that MDC values are not namespaced under labels anymore. I would even go as far to say that #67 was a very bad change, or at the very least a shortsighted one.

• It was a breaking change without an option to restore previous behavior.
• Application developers can now easily add fields by mistake that are not defined by the ECS.
• Application developers can accidentally override existing fields. This will regularly happen because developers often just dump a map with meta information about the current context into the MDC.
• Application developers need to prefix the MDC keys with labels. or tags. to have the values correctly put into the ECS. This makes no sense at all to me. Our developers don’t even know that their applications are logging in the ECS format. This only happens in production where the logging configuration is injected as external configuration by the DevOps team (aka, me). There is just no way I can reliably nudge all our developers to always prefix all their MDC keys with labels. (nor do I want to, because the applications shouldn’t be developed against a specific logging format).

For the stated reasons, please at least provide an option to restore the previous behavior by making the namespace (and by “namespace” i mean “common key prefix”) for MDC values configurable using the logback.xml file.