ValueError: set_wakeup_fd only works in main thread

Hello everyone! Related to: 132:

Current situation

My environment is Windows Server 2016, Apache 2.4.39, Python 3.8.1 + mod_wsgi 4.7.1 + Django 3.0.3 + asgiref 3.2.3. I just updated from Django 2.2.10 to 3.0.3 and then my project started to crash with traceback:

[2020-02-20 17:58:04] | MainProcess     | ERROR    | 222   | django.request | log_response | Internal Server Error: /api/config/
Traceback (most recent call last):
  File "C:\Program Files\Python38\Lib\site-packages\django\contrib\auth\middleware.py", line 57, in process_request
    username = request.META[self.header]
KeyError: 'REMOTE_USER'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Python38\Lib\site-packages\django\core\handlers\exception.py", line 34, in inner
    response = get_response(request)
  File "C:\Program Files\Python38\Lib\site-packages\django\utils\deprecation.py", line 93, in __call__
    response = self.process_request(request)
  File "C:\Program Files\Python38\Lib\site-packages\DiffApp\core\common\middleware.py", line 15, in process_request
    super(CustomRemoteUserMiddleware, self).process_request(request)
  File "C:\Program Files\Python38\Lib\site-packages\django\contrib\auth\middleware.py", line 62, in process_request
    if self.force_logout_if_no_header and request.user.is_authenticated:
  File "C:\Program Files\Python38\Lib\site-packages\django\utils\functional.py", line 224, in inner
    self._setup()
  File "C:\Program Files\Python38\Lib\site-packages\django\utils\functional.py", line 360, in _setup
    self._wrapped = self._setupfunc()
  File "C:\Program Files\Python38\Lib\site-packages\django\contrib\auth\middleware.py", line 24, in <lambda>
    request.user = SimpleLazyObject(lambda: get_user(request))
  File "C:\Program Files\Python38\Lib\site-packages\django\contrib\auth\middleware.py", line 12, in get_user
    request._cached_user = auth.get_user(request)
  File "C:\Program Files\Python38\Lib\site-packages\django\contrib\auth\__init__.py", line 180, in get_user
    user = backend.get_user(user_id)
  File "C:\Program Files\Python38\Lib\site-packages\django\contrib\auth\backends.py", line 161, in get_user
    user = UserModel._default_manager.get(pk=user_id)
  File "C:\Program Files\Python38\Lib\site-packages\django\db\models\manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "C:\Program Files\Python38\Lib\site-packages\django\db\models\query.py", line 411, in get
    num = len(clone)
  File "C:\Program Files\Python38\Lib\site-packages\django\db\models\query.py", line 258, in __len__
    self._fetch_all()
  File "C:\Program Files\Python38\Lib\site-packages\django\db\models\query.py", line 1261, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "C:\Program Files\Python38\Lib\site-packages\django\db\models\query.py", line 57, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "C:\Program Files\Python38\Lib\site-packages\django\db\models\sql\compiler.py", line 1142, in execute_sql
    cursor = self.connection.cursor()
  File "C:\Program Files\Python38\Lib\site-packages\django\utils\asyncio.py", line 19, in inner
    event_loop = asyncio.get_event_loop()
  File "C:\Program Files\Python38\lib\asyncio\events.py", line 636, in get_event_loop
    self.set_event_loop(self.new_event_loop())
  File "C:\Program Files\Python38\lib\asyncio\events.py", line 656, in new_event_loop
    return self._loop_factory()
  File "C:\Program Files\Python38\lib\asyncio\windows_events.py", line 310, in __init__
    super().__init__(proactor)
  File "C:\Program Files\Python38\lib\asyncio\proactor_events.py", line 632, in __init__
    signal.set_wakeup_fd(self._csock.fileno())
ValueError: set_wakeup_fd only works in main thread

If I reboot Apache several time and if I lucky enough I can enter the proper thread (is is my suggestions) and app is running fine.

Now I have the temporal solution which is to add following lines to asgiref\__init__.py (as it was suggested in 132:

if sys.platform == "win32" and sys.version_info >= (3, 8, 0):
    asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

[GrahamDumpleton]

FWIW, there will be a more permanent workaround for this CPython behaviour in mod_wsgi version 4.8.0 when released.

[GrahamDumpleton]

I would suggest in mod_wsgi config, if on Windows, of using something like:

WSGIScriptAlias / /some/path/wsgi.py application-group=%{GLOBAL}

The use of application-group on WSGIScriptAlias will force pre-loading of the WSGI script file. This preloading should occur in the main thread.

What I suspect may be occurring is that the threading module isn’t being imported during normal Python interpreter initialisation, and is only subsequently being imported the first time a request arrives, which will be on an external thread (non main thread), as a result the main_thread() gets initialised to be the external thread and something blows up.

Anyway, this is guess as I don’t grok the overall issue as brain not working well enough to get my head around it.

If this solves the problem, I would suggest this is a bug in CPython in that the threading module assumes that it is always first imported by the main thread and never another thread.

# Create the main thread object,
# and make it available for the interpreter
# (Py_Main) as threading._shutdown.

_main_thread = _MainThread()

...

def main_thread():
    """Return the main thread object.

    In normal conditions, the main thread is the thread from which the
    Python interpreter was started.
    """
    return _main_thread

This is fine if the threading module is always imported during interpreter/sub interpreter initialisation, but would be a problem if it isn’t guaranteed to be.