question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Documentation for using find_packages() and package_dir() might be wrong.

See original GitHub issue

In the docs it says under namespace packages it says:

setup(
      name="namespace.mypackage",
      version="0.1",
      package_dir={'': 'src'},
      packages=find_namespace_packages(where='src')
)

When I adapted this for my project I found that I had to change it slightly pointing explicitly to the __int__.py making name the name of package_dir explicit. Failure to do this led to Python files in the package’s top directory being excluded.

This is what I ended up with:

setup(
    name='mypackage',
    packages=find_packages('mypackage/__init__.py'),
    package_dir={'mypackage':'mypackage'},
)

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

8reactions
pgansslecommented, Nov 2, 2018

@Themanwithoutaplan I think you possibly misunderstand what find_namespace_packages are for. It is for adding packages in a Python 3 implicit namespace, so namespace.mypackage. There should be no files in namespace, just packages. e.g.:

$ tree .
.
├── setup.py
└── src
    └── namespace
        └── mypkg
            └── __init__.py

3 directories, 2 files

Using that configuration, I am able to install namespace.mypkg using this setup.py:

from setuptools import setup, find_namespace_packages

setup(
      name="namespace.mypkg",
      version="0.1",
      package_dir={'': 'src'},
      packages=find_namespace_packages(where='src')
)

Which is exactly what we’re expecting. I think your second example is not correct either. If you do not have a namespace package and want to use the src/ layout, lay out your code like this:

$ tree .
.
├── setup.py
└── src
    └── mypkg
        └── __init__.py

2 directories, 2 files

And then your setup.py looks like this:

from setuptools import setup, find_packages

setup(
      name="mypkg",
      version="0.1",
      package_dir={'': 'src'},
      packages=find_packages(where='src')
)

I personally recommend the src layout using a setup.cfg instead of setup.py, like so:

[metadata]
name="mypkg"
version="0.1"

[options]
package_dir=
    =src
packages=find:

[options.packages.find]
where=src

Then your setup.py looks like this:

import setuptools
setuptools.setup()

Good luck!

0reactions
aadamsoncommented, Apr 26, 2019

@pganssle Perhaps I’m misunderstanding something, but I’m trying to do something similar to what you’re describing (create a setuptools package with a namespace package in a directory somewhere below the root of the project), and as a sanity check I’m trying to run your example. It’s not installing as expected (this is on Python 3.5.6):

[adamson@home] {build-test} ~/test-pkg$ tree --charset unicode .
.
|-- setup.py
`-- src
    `-- namespace
        `-- mypackage
            |-- __init__.py
            `-- mod1.py

3 directories, 3 files
[adamson@home] {build-test} ~/test-pkg$ python setup.py install --record files.txt
running install
running bdist_egg
running egg_info
creating src/namespace.mypackage.egg-info
writing src/namespace.mypackage.egg-info/PKG-INFO
writing top-level names to src/namespace.mypackage.egg-info/top_level.txt
writing dependency_links to src/namespace.mypackage.egg-info/dependency_links.txt
writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt'
package init file 'src/namespace/__init__.py' not found (or not a regular file)
reading manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt'
writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/namespace
creating build/lib/namespace/mypackage
copying src/namespace/mypackage/mod1.py -> build/lib/namespace/mypackage
copying src/namespace/mypackage/__init__.py -> build/lib/namespace/mypackage
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/namespace
creating build/bdist.linux-x86_64/egg/namespace/mypackage
copying build/lib/namespace/mypackage/mod1.py -> build/bdist.linux-x86_64/egg/namespace/mypackage
copying build/lib/namespace/mypackage/__init__.py -> build/bdist.linux-x86_64/egg/namespace/mypackage
byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/mod1.py to mod1.cpython-35.pyc
byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/__init__.py to __init__.cpython-35.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying src/namespace.mypackage.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/namespace.mypackage.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/namespace.mypackage.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/namespace.mypackage.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/namespace.mypackage-0.1-py3.5.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing namespace.mypackage-0.1-py3.5.egg
Removing /nas/dft/ire/adamson/.conda/envs/build-test/lib/python3.5/site-packages/namespace.mypackage-0.1-py3.5.egg
Copying namespace.mypackage-0.1-py3.5.egg to /nas/dft/ire/adamson/.conda/envs/build-test/lib/python3.5/site-packages
namespace.mypackage 0.1 is already the active version in easy-install.pth

Installed /nas/dft/ire/adamson/.conda/envs/build-test/lib/python3.5/site-packages/namespace.mypackage-0.1-py3.5.egg
Processing dependencies for namespace.mypackage==0.1
Finished processing dependencies for namespace.mypackage==0.1
writing list of installed files to 'files.txt'
[adamson@home] {build-test} ~/test-pkg$ python -c 'import namespace.mypackage'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named 'namespace'
[adamson@home] {build-test} ~/test-pkg$ cat setup.py 
from setuptools import setup, find_namespace_packages

setup(name="namespace.mypackage",
      version="0.1",
      package_dir={'': 'src'},
      packages=find_namespace_packages(where='src'))
Read more comments on GitHub >

github_iconTop Results From Across the Web

Knowledge Bits — Common Python Packaging Mistakes
An overview of common mistakes made in creating & building a Python package and how to avoid them.
Read more >
Python setuptools: package directory does not exist
I suspect that it has to do with the src directory, as specified in the find_packages(where="./src") call in the setup.py . However, I've...
Read more >
Package Discovery and Namespace Packages - Setuptools
Finding simple packages#. Let's start with the first tool. find: ( find_packages() ) takes a source directory and two lists ...
Read more >
2. Writing the Setup Script — Python 3.11.1 documentation
Package data can be added to packages using the package_data keyword argument to the setup() function. The value must be a mapping from...
Read more >
Build systems - pybind11 documentation
A helper file is provided with pybind11 that can simplify usage with setuptools. ... build ParallelCompile("NPY_NUM_BUILD_JOBS").install() setup(...).
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found