Pytest and caplog doesn't work if InterceptHandler exists

I would like route all logs through loguru and configure them as I see fit, and for that I have created an InterceptHandler as descriped here. I also want to test the output of some of my logs via the caplog trick described here.

It seems that I can’t have both at the same time, as it creates a hanging loop.

Is there another way to test the contents of the logs when using an InterceptHandler ?

[Delgan]

Thanks for the reproducible example, @mehd-io! It’s all clear now.

Basically, Loguru’s handlers are made thread-safe by using a Lock to protect concurrent access of sinks. While processing a log message, the handler first acquires the Lock and release it once the message is logged. If another message tries to be logged at the same time, it won’t be handled and will wait until the Lock is released.

You’re facing a dead-lock due to recursive call of logger.log(). It is very similar to the following scenario:

from loguru import logger

def my_custom_sink(message):
    logger.debug("Processing the message...")
    print(message, end="")

logger.add(my_custom_sink)

logger.info("Hanging...")

The program is hanging because the debug() call is waiting for the Lock acquired by info() to be released, which never happens because my_custom_sink() can’t terminate.

In your case, an InterceptHandler() is attached to the standard root logger. Every message logged using the standard logging library will also be processed by Loguru added handlers. That’s fine, but there’s also a PropagateHandler() which causes every Loguru message to be propagated to standard logging.

Consequently, when logger.info() is used, the Lock of the PropagateHandler() is acquired, then the log message is sent to the logger returned by logging.getLogger(record.name) which itself call the emit() function of the InterceptHandler(). The latter will tries to call logger.log() but will hang forever because the Lock is still not released.

If this wasn’t hanging you would face infinite recursion anyway.

I never realized it, but InterceptHandler and PropagateHandler don’t mix well.

The solution is to find a trick so that the InterceptHandler does not process the messages generated by the PropagateHandler. I can suggest the following updated handlers:

class PropogateHandler(logging.Handler):
        def emit(self, record):
            record.from_loguru = True
            logging.getLogger(record.name).handle(record)

class InterceptHandler(logging.Handler):
    def emit(self, record):
        # Ignore messages from PropagateHandler to avoid hanging
        if getattr(record, "from_loguru", False):
            return

        # Get corresponding Loguru level if it exists
        try:
            level = logger.level(record.levelname).name
        except ValueError:
            level = record.levelno

        # Find caller from where originated the logged message
        frame, depth = logging.currentframe(), 2
        while frame.f_code.co_filename == logging.__file__:
            frame = frame.f_back
            depth += 1

        logger.opt(depth=depth, exception=record.exc_info).log(level, record.getMessage())

I’m not entirely satisfied with this solution, but this is the best I’ve found so far.

[mehd-io]

I confirm that the workaround works 🎉 , thanks @Delgan for the awesome support & library 🙌