Strange interaction between CTRL+C and `CancelScope.cancel`

Here is the repro code:

import anyio

# Try to set this to False
FINAL_CANCEL = True

async def main() -> None:
    while True:
        print(f"=== Start of iteration ===")
        try:
            async with anyio.create_task_group() as tg:
                try:
                    print("Sleep")
                    await anyio.sleep(10)
                except BaseException as exc:
                    print(f"Sleep interrupted by: {type(exc)}")
                    raise
                finally:
                    if FINAL_CANCEL:
                        tg.cancel_scope.cancel()
        except BaseException as exc:
            print(f"Task group raised: {type(exc)}")
            raise

anyio.run(main)

When I set FINAL_CANCEL = True, I get the following output:

=== Start of iteration ===
Sleep
^CSleep interrupted by: <class 'asyncio.exceptions.CancelledError'>
=== Start of iteration ===
Sleep
^CTraceback (most recent call last):
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1854, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/local/lib/python3.9/selectors.py", line 469, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/projects/stork-test/anyio_bug.py", line 27, in <module>
    anyio.run(main)
  File "/projects/stork/.venv/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 55, in run
    return asynclib.run(func, *args, **backend_options)  # type: ignore
  File "/projects/stork/.venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 232, in run
    return native_run(wrapper(), debug=debug)
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 47, in run
    _cancel_all_tasks(loop)
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 63, in _cancel_all_tasks
    loop.run_until_complete(
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1854, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/local/lib/python3.9/selectors.py", line 469, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt

Note that we get two iterations and that I have to press CTRL+C twice before the program exits. Also note that the TaskGroup silences the CancelledError.

---

When I set FINAL_CANCEL = False, I get the expected output:

=== Start of iteration ===
Sleep
^CSleep interrupted by: <class 'asyncio.exceptions.CancelledError'>
Task group raised: <class 'asyncio.exceptions.CancelledError'>
Traceback (most recent call last):
  File "/projects/stork-test/anyio_bug.py", line 27, in <module>
    anyio.run(main)
  File "/projects/stork/.venv/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 55, in run
    return asynclib.run(func, *args, **backend_options)  # type: ignore
  File "/projects/stork/.venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 232, in run
    return native_run(wrapper(), debug=debug)
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 629, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 596, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 1854, in _run_once
    event_list = self._selector.select(timeout)
  File "/usr/local/lib/python3.9/selectors.py", line 469, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt

A single iteration and I only have to press CTRL+C once. Note that in this case the TaskGroup propagates the CancelledError.

I use Python 3.9.2 and the latest anyio master (3a7b67ac269022b0beca06fd8074459db7cf58b4)

[agronholm]

Yeah, you are reraising it, sorry. Ctrl+C handling on asyncio is kinda chaotic in that it cancels all tasks at the same time which wreaks havoc with SC. I will take an in-depth look at this later.

[frederikaalund]

Thanks for taking a look at this.

For what reason are you cancelling the scope in the finally clause?

Good question. In practice, I just wanted to run something “in the background”. Something like this:

@asynccontextmanager
async def run_in_background(coro):
    async with anyio.create_task_group() as tg:
        # Run in background
        tg.start_soon(coro)
        try:
            yield
        finally:
            tg.cancel_scope.cancel()

Reflecting on this, I guess it’s more appropriate to simply do:

        # Run in background
        tg.start_soon(coro)
        yield
        tg.cancel_scope.cancel()

This way, we only cancel the scope on on “normal” exit (and we avoid the whole suppressed exception situation to begin with).

Do you think it should behave differently?

I guess we’re limited by the CTRL+C behaviour of the asyncio backend. IMHO, asyncio should raise KeyboardInterrupt in the running task instead of CancelledError. I realize that anyio can’t do anything about that design choice.

Only work-around that I can think of is to have anyio install a SIGTERM handler and introduce a global SIGTERM_RECEIVED boolean. In turn, CancelScope.__aexit__ always propagates exceptions if SIGTERM_RECEIVED is True. It does seem a bit hacky, though.

In any case, thanks for having a look at this issue. I’ll close it for now.