Passing loguru function to multiprocessing

When passing a logger.info into a multiprocessing.Pool, like as following:

    with multiprocessing.Pool(1) as pool:
        for my_id in range(5):
            pool.apply_async(do_something, (my_id, logger.info))
        pool.close()
        pool.join()

the following exception occurs:

Process ForkPoolWorker-1:
Traceback (most recent call last):
  File "/home/enelson/.pyenv/versions/3.7.3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/home/enelson/.pyenv/versions/3.7.3/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/home/enelson/.pyenv/versions/3.7.3/lib/python3.7/multiprocessing/pool.py", line 110, in worker
    task = get()
  File "/home/enelson/.pyenv/versions/3.7.3/lib/python3.7/multiprocessing/queues.py", line 354, in get
    return _ForkingPickler.loads(res)
AttributeError: 'Logger' object has no attribute 'log_function'

A fully reproducible test case is here

This is using Python 3.7.3 via pyenv in Ubuntu 18.04

[Delgan]

Ok, I fixed this by refactoring the logger and removing closure functions generated by _make_log_function().

The fix will be available in the next v0.3.0 release, thanks to both of you!

[AnesBenmerzoug]

I managed to reduce the test case to just this:

import pickle
from loguru import logger

pickle.loads(pickle.dumps(logger.info))

or this (to mimic more closely what is happening inside apply_async):

from multiprocessing import SimpleQueue
from loguru import logger

q = SimpleQueue()
q.get(q.put(logger.info))

It seems apply_async puts the given arguments pickled by a ForkingPickler in a multiprocessing.SimpleQueue object.