question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Mount hangs when `cli` called with `asyncio.run_in_executor`

See original GitHub issue

I have been experimenting with ratarmount and mostly working nicely so far, thanks

A problem i have hit and struggling to figure out why is that when i call cli(args) directly from my code mounting works as expected

When i shift the same code to run under asyncio.run_in_executor the mountpoint is created but the program hangs as if its running in the foreground

When i control-C my escape hatch unmounts successfully as expected

Im wondering if there is some tty detection or similar going on in the fuse lib, and whether there is a way to force return/backgrounding

Possibly related #76

sys info:

fuse/stable,now 2.9.9-5 amd64 [installed]
#1 SMP Debian 5.10.113-1 (2022-04-29) GNU/Linux
ratarmount==0.11.3 
PYTHON_VERSION = "3.10.2"

Issue Analytics

  • State:open
  • Created a year ago
  • Comments:12 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
mxmlnkncommented, Jul 8, 2022

Btw, I’m not sure if it applies to your use case but you could also avoid forking into the background by specifying --foreground and then keep that process or thread in the background yourself. That way it would also be correctly closed on exit.

You could also use ratarmount as a library but that might require more adaption in your code.

0reactions
phlaxcommented, Jul 8, 2022

possibly related signals http://curiousthing.org/sigttin-sigttou-deep-dive-linux

ive seen a few mentions around mac and linux backgrounding issues <> SIGTTOU

its not clear whether this is the issue or how to resolve, but seems related

working on a minimal repro i realized a few things 8/

firstly it is sys.exiting if successful afaict - when i was thinking my script was successful it was just exiting after mounting the directory

not sure exactly how this is being triggered - i tried both catching SystemExit and adding atexit.register(fun) but neither catch the signal

the only time this doesnt happen is if there is failure, from which im deducing rightly/wrongly that something low-level in FUSE is bypassing the normal python lifecycle

anyhow, exit issues aside - the following code mounts in the background:


import pathlib
import tarfile
import tempfile
import ratarmount


def create_tarball(output):
    with tempfile.TemporaryDirectory() as tmp:
        pathlib.Path(tmp).joinpath("foo.txt").write_text("BAR")
        with tarfile.open(output, "w:gz") as tar:
            tar.add(tmp, arcname=".")
    return output


def mount_dir(tarball):
    ratarmount.cli((tarball, "tmpmount"))


tarball = create_tarball("baz.tar.gz")
mount_dir(tarball)

but the asyncio/executor version sits in the foreground without exiting:


import asyncio
import concurrent


async def mount_dir_async(tarball):
    asyncio.get_event_loop().run_in_executor(
        concurrent.futures.ProcessPoolExecutor(),
        mount_dir,
        tarball)


asyncio.run(mount_dir_async(tarball))

Read more comments on GitHub >

github_iconTop Results From Across the Web

Issues · mxmlnkn/ratarmount - GitHub
Mount hangs when cli called with asyncio.run_in_executor. #93 opened on Jul 7 by phlax · 12. Performance in multi-process setup performance Something is ......
Read more >
Cancel run_in_executor coroutine from the main thread not ...
I know we dont need to cancel the if we have an event flag, but I saw this example from a different answer....
Read more >
Asynchronously Opening and Closing Files in Asyncio
If a file operation takes a long time, perhaps because the file is on a network mount, then the entire Python process will...
Read more >
Concurrency and async / await - FastAPI
Modern versions of Python have support for "asynchronous code" using something called "coroutines", with async and await syntax. Let's see that phrase by...
Read more >
Developing with asyncio — Python 3.11.1 documentation
Debug Mode: By default asyncio runs in product... ... Calling loop.set_debug() . ... One way of doing that is by using the -W...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found