Standard project structure not specified or not easily findable

• I have searched the issues of this repo and believe that this is not a duplicate.

Issue

When Poetry is unable to determine the packages of the current project (folder). It prints an error such as:

[ModuleOrPackageNotFound]
No file/folder found for package <package>

When a user then tries to find something about this error on the web he will find some closed issues talking about the packages portion of the pyproject.toml. There and in some other locations (e.g. GitHub) you can find “If your project structure differs from the standard one supported by poetry…”. However, it is not defined (or will not pop up in search results) defined, as far as I can see, what exactly this standard structure might be.

By looking at https://python-poetry.org/docs/basic-usage/ one might guess what it could be.

[egnor]

For a project like this, clear documentation is almost as important as working functionality. Documentation that says “If your project structure differs from the standard one supported by poetry” without ever once explaining the “standard one supported by poetry” seems like clearly buggy documentation. (Very much understood that bugs and work need prioritizing and not everything can be fixed right away, but the first step is for us all to realize that this is a bug as serious as a code bug.)

I feel like there used to be a page explaining the project layouts supported by Poetry. Am I imagining things?

My current understanding is:

• A single Poetry project (pyproject.toml file) describes ONE distribution package (file to upload to pypi, etc), which may contain some number of import packages (importable directories of code) and/or modules. If you want to build more than one distribution package, you will need multiple pyproject.toml files, which need to be in different directories.
• By default, Poetry assumes that the distribution package contains one top-level import package with the same name as the distribution package, as is conventional but not universal. (There’s some name mangling that happens here. I agree that is Not Cool especially if undocumented.)
• For such conventional distribution packages, Poetry uses default algorithms to try to autodetect where the import package’s code lives (in package_name/…, or src/package_name/…, or ???)
• When the assumption above is not true, the tool.poetry.packages value in pyproject.yaml describes where Python files live and how they should be arranged inside the distribution package, thus defining the import packages/modules assembled there.
• In any case, “poetry install” will effectively assemble the distribution package and install it in the virtualenv. (No actual package file is created; the source files are linked into the virtualenv in “editable mode”, so edits in the source tree take immediate effect.)

In my humble opinion, this all makes reasonable sense, but is very badly underdocumented, leading to confusion and anger:

• Poetry’s treatment and assumptions about distribution packages, import packages, and modules is never made clear; the word “package” is often used in an ambiguous and confusing way.
• The default algorithms used to guess and detect how source files are laid out are completely undescribed.
• The name mangling used when converting the distribution package name into a default import package name is completely undescribed.
• The impact of the “–src” flag to “poetry new” is underdescribed. The project layout related choices made in “poetry init” are underdescribed.
• The exact specification and semantics of tool.poetry.packages value is underdescribed. A couple examples are given but it is not clear exactly what the full range is and what effect it has (e.g. what glob values are allowed).

I’m heartened that @HallerPatrick has started a PR, though it only takes a little nibble at this issue and also seems stalled.

[ghost]

@egnor 's comment above is on point with the issue at hand, and gives a good explanation of what is wrong. What I think should be done to fix the documentation is to include a section within the ‘Basic Usage’ section that explicitly outlines the intended folder structure for the package, even if that is redundant with other documentation.

This means briefly explaining the distinction between package vs. project, breaking down where exactly key files should be in relation to eachother (ie, “your entry point should be in a package folder directly underneath where pyproject.toml is located”), and more clearly reference the documentation in itself (ie, the “Basic Usage” chapter mentions pyproject.toml, but it is not a link, leaving the user to navigate through documentation manually. This is bad practice.) Furthermore, within the pyproject.toml.

The reason for the redundancy and clarifications is that the tone of the documentation assumes a certain degree of pre-existing knowledge of python packaging that just doesn’t exist in the community. As others have stated, packaging in python is notoriously obtuse, poorly-documented, and complex, and the PEPs themselves don’t provide enough documentation to either establish best practice or even a competent baseline understanding of the process. In addition to that, documentation that assumes advanced prior knowledge is just plain bad documentation.

This aspect of documentation shows an area where it just is not explicit enough and not every user of poetry has the time to investigate the source files to find a solution to what should be a simple problem.

In my personal experience, I cannot get my program to run because I keep getting ModuleNotFoundError which is directly caused by me not being able to figure out how exactly Poetry wants tool.poetry.packages and tool.poetry.scripts to be formatted. Yes, the syntax is there, but there is no documentation in any resource, poetry or Python itself which explains “tool.poetry.packages expects your entry point to be in this particular folder relative to pyproject.toml” or “tool.poetry.scripts looks through the listed packages in this manner to find the entry point you declared”.

Finally, there is one more addition that could improve this problem: poetry check could be improved by testing the declared scripts entry point for ModuleNotFoundError, and return a warning that the entry point may not work.