Why is the .egg-info directory of a source distribution ignored when building a wheel using pip?

[thrau]

When I build a source distribution from my project, it will contain an .egg-info directory as part of the .tar.gz. When I then pip install ..., pip creates a bunch of temporary directories, including the original source distribution, as well as temp folders to create the modern .dist-info metadata. to do that, pip calls prepare_metadata_for_build_wheel on the build backend, which in setuptools appears to be just setup.py dist_utils (and first setup.py egg_info).

Consequently, the .egg-info directory that was packaged with the source distribution, including entry_points.txt and other metadata, is completely ignored, since setuptools builds it in the temporary directory from scratch.

Here's a screenshot from a debugging session of pip install <source-distribution> (where source-distribution is just a test library that i created locally), where I captured the pip build context after the wheel was created but before resources were cleaned up.

Is this intentional? Why is the .egg-info packaged into a source distribution in the first place if it's not re-used?

[thrau]

FYI: I noticed this when working on localstack/plux#26 which surfaced an issue after #4647. perhaps @pelson as author or @jaraco @abravalheri as reviewers know more?

[abravalheri]

Consequently, the .egg-info directory, including entry_points.txt and other metadata, is completely ignored, since setuptools builds it from scratch.

.egg-info is not completely ignored, at least not always. In some cases, it is used to bootstrap entry-points (for some packages that is important).

Other than that, the long term objective is to obviate egg_info in favour of a standalone definition of dist_info. But the road to that is long, because the egg_info command is intertwined in the existing code base. PRs helping to achieve that goal are welcome.

[thrau]

.egg-info is not completely ignored, at least not always

whatever lands in the source distribution's .egg-info directory may be used or not, as shown in my screenshot, correct? both the already compiled entry_points.txt and the custom plux.json are not used, since a new .egg-info is created from the project configuration.

i don't understand why that is the case? why create a new .egg-info in the first place when the source distribution already comes with one?

in favour of a standalone definition of dist_info

how will this affect source distributions? will .dist-info directories be shipped with the source distribution instead of .egg-info? or is the assumption that the .dist-info directory can always be created during setup time from the python project config?

[abravalheri]

The already compiled entry_points.txt may be used in some cases, depending on the project layout. In the sense that it will be picked up when traversing the entry-points during the build. That maybe be used by some projects to trigger plugins and customisations. Some tools may also read requires.txt, since PKG-INFO did not use to contain the dependency list. If I remember correctly setuptools itself also reads SOURCES.txt.

The files are regenerated because it is easier to deal with them in that way given the existing code base.

The idea of a standalone .dist-info is that it would not be generated using .egg-info as an intermediary step, but directly from the relevant files.