Conversation
Currently, there are two main options for a project layout, as described in https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/. `<package_name>/__init__.py`: The project is importable even when not installed. The community seems to be migrating to the `src` layout https://hynek.me/articles/testing-packaging/. `src/<package_name>/__init__.py`: We add an extra layer of directory only to repeat the package name. This makes the library layout feel higher overhead than needed, making adoption especially in small projects harder, where the alternative is just having some Python files in a directory without any packaging. As an alternative, we introduce `src/__init__.py`, a layout familiar from other programming languages. It is low overhead (no extra indirection, docs are "put your python files in `src`, `uv sync` and `import <package_name>`") and it enforces isolation through (editable) installation (you can't `import <package_name>` directly). Python does not support this natively it always want the module name in the directory or filename. We hack around this by using a custom meta finder that we insert at the top import priority. This code runs at every python startup. Imports are lazy where possible to reduce overhead. Since we're hacking the Python import system to achieve this, it's experimental. I like `src/__init__.py` a lot, but everything that runs code through `.pth` in implicit, pre-main life is to be regarded with some suspicion. This PR only adds editable support (the harder part), proper `build_{sdist,wheel}` will be added in an upstack PR.
| if preview.is_disabled() { | ||
| warn_user_once!( | ||
| "The direct src layout is experimental and may be removed without warning" | ||
| ); | ||
| } |
There was a problem hiding this comment.
This feature should be stabilized separately from the general build backend.
|
In this proposed layout, what determines the name of your top-level importable package name? It is determined by the name of the outer directory containing the Regardless of the answer, I find this pretty confusing. Why should the top-level package have an intervening Also, this layout doesn't really address the core problem with the "flat" layout that the "src" layout was originally meant to address, that if you run What concrete advantage does this have over the existing flat layout? Static analysis tools will be unable to understand libraries using this layout, unless we convince them to add special-cased support for it. |
|
Another good post on why src/package_name layout is good: https://blog.ionelmc.ro/2014/05/25/python-packaging/ |
| /// The directory that contains the module directory relative to the project root. | ||
| /// | ||
| /// The default value is `src`, for the `src/<package_name>/__init__.py` layout. It is an empty | ||
| /// path when using the flat layout `<package_name>/__init__.py` over the src layout. | ||
| pub(crate) module_root: PathBuf, | ||
|
|
||
| /// Enable to direct src layout to enable `src/__init__.py` as sources root. | ||
| /// | ||
| /// The direct src layout is a custom uv extension that avoids nesting in the `src` directory, | ||
| /// instead of `src/<package_name>/__init__.py` or `<package_name>/__init__.py`, it uses | ||
| /// `src/__init__.py`. The module root still needs to be set to `src`. | ||
| /// | ||
| /// Note that this layout always has to go through an installation process, otherwise Python | ||
| /// consider the module name `src`, not `<package_name>` | ||
| pub(crate) direct_src: bool, | ||
|
|
There was a problem hiding this comment.
It somehow confuses me as a user.
module_rootauto appends<package_name>to the given name.direct_srcprevents previous step, then on editable installs, uses_package_name.pthand_package_name_direct_src_support.pyfor custom import loader.
Another approach could be always using direct path for module_root (i.e. src points to src/__init__.py) and always use custom import loader for editable installs.
|
I actually just made a post about this -- I'd love this, personally |
Currently, there are two main options for a project layout, as described in https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/.
<package_name>/__init__.py: The project is importable even when not installed. The community seems to be migrating to thesrclayout https://hynek.me/articles/testing-packaging/.src/<package_name>/__init__.py: We add an extra layer of directory only to repeat the package name. This makes the library layout feel higher overhead than needed, making adoption especially in small projects harder, where the alternative is just having some Python files in a directory without any packaging.As an alternative, we introduce
src/__init__.py, a layout familiar from other programming languages. It is low overhead (no extra indirection, docs are "put your python files insrc,uv syncandimport <package_name>") and it enforces isolation through (editable) installation (you can'timport <package_name>directly).Python does not support this natively it always want the module name in the directory or filename. We hack around this by using a custom meta finder that we insert at the top import priority. This code runs at every python startup. Imports are lazy where possible to reduce overhead. Since we're hacking the Python import system to achieve this, it's experimental. I like
src/__init__.pya lot, but everything that runs code through.pthin implicit, pre-main life is to be regarded with some suspicion.This PR only adds editable support (the harder part), proper
build_{sdist,wheel}will be added in an upstack PR.