ctypes pickling error

Good day, Delgan.

I have an error I am encountering in the handlers of loguru. This error is being raised when I use “logger.exception(msg)” in the except block of a try statement.

Some context: I’m developing an application to interface with NVIDIA’s AI Triton server. The server’s client-side python bindings are written in both C/C++ and Python. I’m mentioning this because I am developing on Windows, and replicating this issue is fairly hard due to specifics in the environment for the Triton server. I can give you the files and procedures to set up an environment like mine for this if you’d like, but I’ll leave that up to you.

The error: Stacktrace:

Traceback (most recent call last):
  File "C:\Users\jules\.conda\envs\app\lib\site-packages\loguru\_handler.py", line 175, in emit
    self._queue.put(str_record)
  File "C:\Users\jules\.conda\envs\app\lib\multiprocessing\queues.py", line 362, in put
    obj = _ForkingPickler.dumps(obj)
  File "C:\Users\jules\.conda\envs\app\lib\multiprocessing\reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
ValueError: ctypes objects containing pointers cannot be pickled

Code throwing the issue:

try:
    # <Code here raises 'InferenceServerException'>
except InferenceServerException: # The specific exception being raised (Triton server-specific)
    logger.exception("An issue occurred while contacting the inference server:\n")

The exception being raised does have calls to dll’s in it, so that could possibly be the root cause? Again, it is difficult to reproduce this error because of the environment.

Exception data as returned by ‘sys.exc_info()’:

type: <class 'tensorrtserver.api.InferenceServerException'>
value: InferenceServerException(c_void_p(2735452554832)) # This is what I think is causing the issue
traceback: <normal traceback>

Logger setup code:

import os
import sys

import stackprinter
from loguru import logger


class FilterLevel(object):
    """Used to filter logs based dynamic logging level by using the
    :prop:`level` property which can be either an :obj:`int` or
    :obj:`str`.
    """

    _level: Union[str, int]

    def __init__(self) -> None:
        super().__init__()

        self.level = "INFO"

    def __call__(self, record: dict) -> bool:
        levelno = logger.level(self.level).no
        return record["level"].no >= levelno

    @property
    def level(self) -> Union[str, int]:
        return self._level

    @level.setter
    def level(self, value: Union[str, int]) -> None:

        if not isinstance(value, (str, int)):
            raise TypeError("property 'level' must be of type 'str' or 'int': "
                            f"'{type(value).__name__}'")

        self._level = value

LOGFILE = "some_log_somewhere.log"
LOGGER_FILTER = FilterLevel()
LOGGER_FORMAT = "<green>{time:YYYY-MM-DD HH:mm:ss.SSS}</green> | <level>{level: <8}</level> | <cyan>{name}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> - <level>{message}</level>"


def _format_stacktrace(record: dict):
    format_ = LOGGER_FORMAT # The default format found in loguru's '_defaults.py'

    if record["exception"] is not None:
        record["extra"]["stack"] = stackprinter.format(record["exception"])
        format_ += "<red><bold>{extra[stack]}</bold></red>\n"

    return format_ + "\n"

# Remove all current handlers
logger.remove()

# Add the needed handlers
logger.add(sys.stderr, format=_format_stacktrace, enqueue=True)
logger.add(LOGFILE, format=_format_stacktrace, filter=LOGGER_FILTER, enqueue=True)

# Create loguru attributes to be accessible from the logger anywhere
# the logger is found object
setattr(logger, "app_logfile", LOGFILE)
setattr(logger, "app_logfilter", LOGGER_FILTER)

Please let me know if there is anything more I can provide. Jules

[pbednarz1]

Any progress on that?

[Delgan]

@holyachon @JulianOrteil This issue is not yet fixed but I’ll handle it soon.