Cannot import file/folder as object

About

Now, many python developers use typing and find circular import error. So we avoid this by import file/folder as object. However, CPython allow this way but Nuitka not allow now.

Problem

• Nuitka 0.6.5
• Python: 3.6.7 64-bit
• Windows x86_64

I try to compile with fastAPI and find error.

Traceback (most recent call last):
  File "C:\Users\nicert\PycharmProjects\bc4py\fast.dist\fast.py", line 1, in <module>
  File "C:\Users\nicert\PycharmProjects\bc4py\fast.dist\fastapi\__init__.py", line 7, in <module fastapi>
  File "C:\Users\nicert\PycharmProjects\bc4py\fast.dist\fastapi\applications.py", line 3, in <module fastapi.applications>
  File "C:\Users\nicert\PycharmProjects\bc4py\fast.dist\fastapi\routing.py", line 8, in <module fastapi.routing>
  File "C:\Users\nicert\PycharmProjects\bc4py\fast.dist\fastapi\dependencies\utils.py", line 23, in <module fastapi.dependencies.utils>
  File "C:\Users\nicert\PycharmProjects\bc4py\fast.dist\fastapi\utils.py", line 5, in <module fastapi.utils>
ImportError: cannot import name 'routing'

FastAPI use this technique to avoid circular imports. problem

Simple example

import_bug
├── __init__.py
├── a.py
└── b.py

# a.py
from import_bug import b
from typing import Optional


class A:
    def __init__(self):
        self.b: Optional[b.B] = None

    def set(self, new):
        self.b = new

# b.py
from import_bug import a
from typing import Optional


class B:
    def __init__(self):
        self.a: Optional[a.A] = None

    def set(self, new):
        self.a = new

# bugger.py
from import_bug.a import A
from import_bug.b import B


if __name__ == '__main__':
    a = A()
    a.set(B())
    print(a, a.b)

compile by python -m nuitka --mingw64 -j 2 --show-progress --show-scons --recurse-all --standalone bugger.py

error

Traceback (most recent call last):
  File "C:\Users\nicert\PycharmProjects\nicert\bugger.dist\bugger.py", line 1, in <module>
  File "C:\Users\nicert\PycharmProjects\nicert\bugger.dist\import_bug\a.py", line 1, in <module import_bug.a>
  File "C:\Users\nicert\PycharmProjects\nicert\bugger.dist\import_bug\b.py", line 1, in <module import_bug.b>
ImportError: cannot import name 'a'

[justin0mcateer]

I am having the same problem with FastAPI. It appears to be a run-time problem with the way the ‘from {module} import {sub-module}’ statement behaves. It does not appear to be a compile time problem. I am seeing that the ‘routing.py’ submodule is being compiled and linked by Nuitka, but the program still complains that it is not able to find it at run-time.

I am also using Uvicorn and I can say that the problem with Uvicorn and the problem with FastAPI are not the same. The Uvicorn issue seems to be solved merely by using ‘include-module’ directives. This is my compile command:

python -m nuitka --standalone --follow-imports --show-progress --show-scons --show-modules --jobs=15 --include-module=uvicorn.logging --include-module=uvicorn.protocols.http --include-module=uvicorn.loops --include-module=uvicorn.lifespan --include-module=uvicorn.protocols.websockets uvicorn_test.py

However, neither the ‘include-module’ nor the ‘include-package’ directives are helpful in solving the FastAPI run-time issue. This is my compile command with FastAPI:

python -m nuitka --standalone --follow-imports --show-progress --show-scons --show-modules --jobs=15 --include-package=pydantic --include-package=fastapi --include-module=uvicorn.logging --include-module=uvicorn.protocols.http --include-module=uvicorn.loops --include-module=uvicorn.lifespan --include-module=uvicorn.protocols.websockets fastapi_test.py

This compiles fine and includes the ‘fastapi.routing’ submodule as demonstrated by the Nuitka and Scons output"

Nuitka:INFO:Included compiled module 'fastapi.routing'.

scons: building `fastapi_test.build/module.fastapi.routing.o' because it doesn't exist
gcc -o fastapi_test.build/module.fastapi.routing.o -c -fvisibility=hidden -std=c11 -fwrapv -fpartial-inlining -ffile-prefix-map=fastapi_test.build/static_src=/home/jmcateer/.local/share/dephell/venvs/nuitka_fastapi-7F4f/main/lib/python3.7/site-packages/nuitka/build/static_src -fno-var-tracking -O3 -pipe -D_NUITKA_STANDALONE -D_NUITKA_SYSFLAG_BYTES_WARNING=0 -D_NUITKA_SYSFLAG_NO_SITE=0 -D_NUITKA_SYSFLAG_VERBOSE=0 -D_NUITKA_SYSFLAG_UTF8=0 -D_NUITKA_SYSFLAG_OPTIMIZE=0 -D__NUITKA_NO_ASSERT__ -D_NUITKA_FROZEN=165 -D_NUITKA_MODULE_COUNT=730 -D_NUITKA_EXE -I/usr/include/python3.7m -I/usr/PC -Ifastapi_test.build -I/home/jmcateer/.local/share/dephell/venvs/nuitka_fastapi-7F4f/main/lib/python3.7/site-packages/nuitka/build/include -I/home/jmcateer/.local/share/dephell/venvs/nuitka_fastapi-7F4f/main/lib/python3.7/site-packages/nuitka/build/static_src fastapi_test.build/module.fastapi.routing.c

The object file also appears in the linker input at ‘@sources.tmp’:

fastapi_test.build/module.fastapi.routing.o

However, when the program is run, it fails immediately on startup with the following error:

ImportError: cannot import name 'routing' from 'fastapi' (/home/jmcateer/Documents/nuitka_fastapi/fastapi_test.dist/fastapi/__init__.py)

@namuyan seems to have worked around this issue by modifying FastAPI. However, this is not an uncommon import pattern and modifying the upstream FastAPI library is not really a good option for me.

Any specific ideas about why this fails and run-time and how we could fix in Nuitka, versus modifying the compiled code?

The test code being compiled is attached. fastapi_test.zip

[kayhayen]

This has been solved through a duplicate, cyclic name imports have been working for a while now.