Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Cannot use loguru handlers with spark due to lru_cache

See original GitHub issue

I am running an application on a spark cluster.

The driver creates a telegram handler and adds it to loguru. In older versions of loguru (e.g. 0.3.2) this was working. In more recent versions of loguru, when the driver starts pickling objects to send them to the executors there are issues with unpickable objects, for example I get this error when adding the telegram handler: Can't pickle <functools._lru_cache_wrapper object at 0x7f3d8688f4b0>: it's not the same object as urllib.parse.urljoin

Issue Analytics

State:
Created 3 years ago
Comments:8 (4 by maintainers)

Top GitHub Comments

1reaction

dzanagacommented, Mar 23, 2021

It works! Thanks for the solution.

0reactions

dzanagacommented, Mar 22, 2021

@Delgan didn’t try yet, will test asap, thanks

Top Results From Across the Web

Code snippets and recipes for loguru

Several solutions are possible, tough, if you need to change the configured level of a handler. Chose the one that best fits your...

How do I log from my Python Spark script - Stack Overflow

Handler interface in a class that forwards log messages to log4j under Spark. Then use logging.root.addHandler() (and, optionally ...

how to get python to log Code Example - Code Grepper

import logging logging.basicConfig(level=logging.DEBUG) # logging.basicConfig(filename='app.log', filemode='w', format='%(name)s - %(levelname)s ...

pkg_desc_index « metadata - repo/sync/gentoo.git

... for users with access to system-wide Bitcoin services acct-group/bitflu 0: ... for netfilter/iptables related logging app-admin/usbview 2.2 3.0: Display ...

main - Anaconda repo

Package Latest Version Doc Dev License linux‑64 osx‑64 win‑64 7za 920 doc LGPL X 7zip 19.00 dev LGPL‑2.1‑or‑later X _anaconda_depends 2022.05 doc dev BSD X X...