shell_plus notebook settings Import error in subdirectory

I have an issue where when I run python manage.py shell_plus --notebook I get an error where it can’t find my Django settings when I navigate and create a notebook in a subdirectory of the project:

WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
[IPKernelApp] WARNING | Error in loading extension: django_extensions.management.notebook_extension
Check your config files in /home/eric/.ipython/profile_default
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/IPython/core/shellapp.py", line 269, in init_extensions
    self.shell.extension_manager.load_extension(ext)
  File "/usr/local/lib/python2.7/dist-packages/IPython/core/extensions.py", line 86, in load_extension
    if self._call_load_ipython_extension(mod):
  File "/usr/local/lib/python2.7/dist-packages/IPython/core/extensions.py", line 133, in _call_load_ipython_extension
    mod.load_ipython_extension(self.shell)
  File "/usr/local/lib/python2.7/dist-packages/django_extensions/management/notebook_extension.py", line 5, in load_ipython_extension
    style=no_style())
  File "/usr/local/lib/python2.7/dist-packages/django_extensions/management/shells.py", line 102, in import_objects
    setup()
  File "/usr/local/lib/python2.7/dist-packages/django/__init__.py", line 17, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 48, in __getattr__
    self._setup(name)
  File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 44, in _setup
    self._wrapped = Settings(settings_module)
  File "/usr/local/lib/python2.7/dist-packages/django/conf/__init__.py", line 92, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
ImportError: No module named MY_DJANGO_APP.settings.common

It works when I create a python notebook from the root directory of my project, but not from any subdirectory.

Any suggestions?

[seanjh]

I managed to work around this problem, in case it’s useful for anyone else. Basically, I added an ipython_config.py file to my notebook sub-directory, then manually inserted my base project directory into sys.path.

Project layout:

/project
    /someapp
    /notebooks
        ipython_config.py
    settings.py

settings.py

NOTEBOOK_ARGUMENTS = [
    '--notebook-dir', 'notebooks',
]

IPYTHON_ARGUMENTS = [
    '--debug',
]

notebooks/ipython_config.py

import sys
import os

FILE_PATH = os.path.abspath(os.path.dirname(__file__))
PROJECT_BASE_PATH = os.path.abspath(os.path.join(FILE_PATH, '..'))

# Allows the kernel to "see" the project during initialization. This
# FILE_PATH corresponds to Jupyter's "notebook-dir", but we want notebooks to
# behave as though they resided in the base directory to allow for clean
# imports.
print("sys.path BEFORE = {}".format(sys.path))
sys.path.insert(1, PROJECT_BASE_PATH)
print("sys.path AFTER = {}".format(sys.path))

# Any additional initialization logic goes here

In the logs, you should observe IPython locating that ipython_config.py file in your subdirectory. For example:

[D 13:28:12.612 NotebookApp] Starting kernel: [u'/usr/bin/python', u'-m', u'ipykernel', u'-f', u'MY_USER_HOME/.local/share/jupyter/runtime/kernel-....json', u'--debug', u'--ext', u'django_extensions.management.notebook_extension']
[D 13:28:12.616 NotebookApp] Connecting to: tcp://127.0.0.1:45969
[I 13:28:12.616 NotebookApp] Kernel restarted: f2c2cfa1-680e-4b17-bd95-03e1affb9b77
[D 13:28:12.617 NotebookApp] Connecting to: tcp://127.0.0.1:52312
[IPKernelApp] IPYTHONDIR set to: MY_USER_HOME/.ipython
[IPKernelApp] Using existing profile dir: u'MY_USER_HOME/.ipython/profile_default'
[IPKernelApp] Searching path [u'MY_PROJECT_BASE_PATH/notebooks', u'MY_USER_HOME/.ipython/profile_default', '/usr/local/etc/ipython', '/etc/ipython'] for config files
[IPKernelApp] Attempting to load config file: ipython_config.py
[IPKernelApp] Looking for ipython_config in /etc/ipython
[IPKernelApp] Looking for ipython_config in /usr/local/etc/ipython
[IPKernelApp] Looking for ipython_config in MY_USER_HOME/.ipython/profile_default
[IPKernelApp] Looking for ipython_config in MY_PROJECT_BASE_PATH/notebooks
sys.path BEFORE = ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/dist-packages/IPython/extensions', 'MY_USER_HOME/.ipython']
sys.path AFTER = ['', MY_PROJECT_BASE_PATH, '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/local/lib/python2.7/dist-packages/IPython/extensions', 'MY_USER_HOME/.ipython']
[IPKernelApp] Loaded config file: MY_PROJECT_BASE_PATH/notebooks/ipython_config.py
[IPKernelApp] Attempting to load config file: ipython_kernel_config.py
[IPKernelApp] Looking for ipython_kernel_config in /etc/ipython
[IPKernelApp] Looking for ipython_kernel_config in /usr/local/etc/ipython
[IPKernelApp] Looking for ipython_kernel_config in MY_USER_HOME/.ipython/profile_default
[IPKernelApp] Looking for ipython_kernel_config in MY_PROJECT_BASE_PATH/notebooks
...
[IPKernelApp] Starting the kernel at pid: 209
...
[IPKernelApp] Loading IPython extensions...
[IPKernelApp] Loading IPython extension: storemagic
[IPKernelApp] Loading IPython extension: django_extensions.management.notebook_extension
[IPKernelApp] 
*** MESSAGE TYPE:kernel_info_request***
[IPKernelApp]    Content: {}
   --->
...

[jonathanstrong]

+1 would love a solution to this problem - i have like 150 notebooks in the root dir of my project, it’s getting out of hand