[21.6.x on pyinstaller builds crash] ModuleNotFoundError: No module named 'websockets.legacy.protocol'

Describe the bug We build our server into a single macOS executable with pyinstaller. sanic works perfectly up until version 21.3.4 After 21.6.0 (including both 21.6.1, 21.6.2)

When the app is run and it’s going through the sanic imports we get the following crash and stacktrace:

Traceback (most recent call last):
  File "telluride.py", line 28, in <module>
    import server
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "server.py", line 23, in <module>
    from sanic import Sanic
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "sanic/__init__.py", line 2, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "sanic/app.py", line 43, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "sanic/asgi.py", line 11, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "sanic/models/asgi.py", line 6, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 546, in exec_module
  File "sanic/websocket.py", line 13, in <module>
  File "websockets/imports.py", line 81, in __getattr__
  File "websockets/imports.py", line 26, in import_name
ModuleNotFoundError: No module named 'websockets.legacy.protocol'
[32682] Failed to execute script 'telluride' due to unhandled exception!

sanic/websocket.py:13

Code snippet Our tiny http server code here

Expected behavior Not crash, the same way it works with all previous versions of sanic up to 21.3.4

Environment (please complete the following information):

• OS: macOS (so far, still need to test in windows and linux, will follow up on this thread in a few minutes)
• Version 21.6.0, 21.6.1, 21.6.2
• The error comes up with both python 3.7 and 3.9.6

[ahopkins]

I am sorry you have been struggling. You never messaged me about this so I would have assumed it was resolved. I wish you would have reached out sooner and I could have helped you through this rather than struggling with frustration. Your dependency issue seems to be resolved, and I think this new error is potentially easy to overcome.

You are running Sanic in a capacity that is not standard. PyInstaller attempts to do a lot of code stripping. Sanic itself tries to do this at startup to optimize your performance. So, what is happening is that PyInstaller is removing parts of Sanic. When the Sanic server starts, it throws an OSError because the code that it thinks should be available does not exist.

I just went through and put the following snippet into a fresh venv with Sanic and PyInstaller.

$ pip install PyInstaller sanic
...

$ pip freeze                                         
aiofiles==0.8.0
altgraph==0.17.2
httptools==0.3.0
multidict==5.2.0
pyinstaller==4.8
pyinstaller-hooks-contrib==2021.5
sanic==21.12.1
sanic-routing==0.7.2
ujson==5.1.0
uvloop==0.16.0
websockets==10.1

Then, I created a simple script that approximates the code you shared:

import asyncio
from multiprocessing import Process, freeze_support

from sanic import Sanic, json

DEFAULT_HTTP_PORT = 9999
BUILD = 123
SERVER_PORT = DEFAULT_HTTP_PORT


class SomeServer:
    @staticmethod
    async def start(
        build_number, http_port_number=DEFAULT_HTTP_PORT, workers_number=1
    ):
        app = Sanic(f"TellurideWebServer_{build_number}")

        @app.route("/")
        async def root_handler(request):
            return json(
                {
                    "build": build_number,
                    "message": "no valid parameters received",
                }
            )

        app.run(
            host="127.0.0.1", port=http_port_number, workers=workers_number
        )


def async_main():
    asyncio.run(main())


async def main():
    server = SomeServer()
    await (server.start(BUILD, SERVER_PORT, 2))


if __name__ == "__main__":
    freeze_support()
    Process(target=async_main).start()

I was able to recreate your issue:

$ python -m PyInstaller /tmp/issue2227.py
...

$ ./dist/issue2227/issue2227
...
OSError: could not get source code
multiprocessing/process.py:331: RuntimeWarning: coroutine 'Loop.create_server' was never awaited

A quick look at the PyInstaller docs showed me this:

–collect-all MODULENAME

Collect all submodules, data files, and binaries from the specified package or module. This option can be used multiple times.

A quick rebuild:

$ python -m PyInstaller /tmp/p.py --collect-all=sanic

Sure enough, when I run the executable now, it runs as expected and I can ping the endpoint:

$ curl localhost:9999
{"build":123,"message":"no valid parameters received"}

---

I hope this helps. If there are more people looking to make a PyInstaller executable with Sanic perhaps we can add it to the How to... section of the documentation.

[gubatron]

Wow, thank you Adam for taking the time to look at this in detail, wish I’d found that --collect-all flag for PyInstaller earlier!

Hopefully this will help someone that might have a similar problem in the future.