Custom Properties/Dimensions Design Discussion

I’m happy to help contribute the code, but wanted to share design first and discuss to see if it is right approach.

Is your feature request related to a problem?

Support for custom properties/dimensions was recently added for logging messages using args.

logger.info("my message", {"customProperty": "myValue"})

There were a few concerns with the current approach when applying it to existing code:

1. Current approach with *args breaks existing logging functionality for other registered logging handlers, which are using args for lazy string interpolation. We would need to go back and change existing log calls to remove existing args (interpolate strings first) and change arg type to dictionary, if we wanted to log custom properties.
2. It breaks LoggerAdapter and filter patterns for passing contextual information to all messages. For LoggerAdapter pattern, we cannot modify args at all. For filter pattern, we would currently need to overwrite the args there.
3. We need to pass the custom dimensions into every call to the logger because of concern number 2.
4. It only supports messages and not metrics/exceptions, so approach is not consistent.

Describe the solution you’d like.

Instead of using the *args as a dict, I would suggest using the **kwargs approach.

For the logging library, **kwargs are added to the dict of LoggerRecord. When we process the record, we could look for customDimensions to be present on the record as an attribute. If it is there, then add it to the properties to be sent to Application Insights.

This solution would leave in place existing behavior, since end users can still use kwargs for storing other important attributes as long as it does not conflict with customDimensions (which is very application insights specific term).

As part of this request, it would also be great to have consistency across messages, exceptions, events, etc, instead of only supporting custom properties for messages.

Below are examples of logging use cases with the suggested approach:

Example 1: Simple Case for Adding Properties Once

properties = {‘customDimensions’: {‘testKey’: ‘testValue’}}
logger.warning(“Starting retry %s”, retries, properties)

Example 2: Pass Custom Properties as Context with LoggerAdaptor pattern

# FYI, just quickly sketching out code for explaining, more checking is needed for nulls, etc.
class CustomAdapter(logging.LoggerAdapter):
    def process(self, msg, kwargs):
	kwargs['customDimensions'].update(self.extra[‘customDimensions’])
        return msg, kwargs

logger = logging.getLogger(__name__)
adapter = CustomAdapter(logger, {‘customDimensions’: {‘testKey’: ‘testValue’}})
adapter.warning(‘hello’)
adapter.info(‘hello %s’, name, {‘customDimensions’: {‘optionalExtraKey’: ‘optionalExtraValue’}})

Example 3: Pass Custom Properties as Context with LoggerFilter pattern

# FYI, just quickly sketching out code for explaining, more checking is needed for nulls, etc.

class ContextFilter(logging.Filter):

    CUSTOM_DIMENSIONS = {‘testKey’: ‘testValue’}

    def filter(self, record):
        setattr(record, ‘customDimensions’, CUSTOM_DIMENSIONS)
	return True

logger = logging.getLogger(__name__)
f = ContextFilter()
logger.addFilter(f)

logger.warning(‘hello’)
logger.info(‘hello %s’, name, {‘customDimensions’: {‘optionalExtraKey’: ‘optionalExtraValue’}})

Describe alternatives you’ve considered.

We tried the new approach with adding properties as args, but it required too much change to existing logs and required passing args to every call. We hope to instead use existing patterns for contextual logging.

[lzchen]

@ctaggart I’ll be cutting a new release at the end of Jan.

[ctaggart]

This one is blocking our use of the library. I would love to see a new build released.