Add build constraints #13534

notatallshaw · 2025-08-09T02:28:46Z

Fixes #13300

This adds a new --build-constraint flag to complement the existing --constraint flag, at a high level:

Build constraints are used as constraints for build requirements in isolated build environments
Regular constraints are no longer used as constraints in any isolated build environments (normally done via PIP_CONSTRAINT now requires PIP_BUILD_CONSTRAINT or simply --build-constraint)
Because this changes existing behavior the new behavior is only enabled when build constraints are passed or the --use-feature flag build-constraint is enabled, as not to break user workflows
There is a deprecation warning when using PIP_CONSTRAINT is present and in effect without using the new build constraints

This change in behavior was desired because:

It is a simpler UX than the existing behavior of specifying both regular and build time constraints (CLI flags for run time and ENV vars for isolated build vs. --constraint and --build-constraint)
This behavior will be easier to match with in-process installation (Install build dependencies in-process #13450) where as the existing behavior is trickier
It is the same behavior uv pip has for --constraint and --build-constraint so is well verified in the real world

src/pip/_internal/build_env.py

…needing to use the `build-constraint` feature

potiuk · 2025-08-11T11:51:14Z

Very, very much needed. Thanks @notatallshaw -> today setuptools-scm 9.1.0 (2 hours ago) again broke source installation of xmlsec: pypa/setuptools-scm#1194

potiuk · 2025-08-11T12:14:02Z

Very, very much needed. Thanks @notatallshaw -> today setuptools-scm 9.1.0 (2 hours ago) again broke source installation of xmlsec: pypa/setuptools-scm#1194

Yanked now luckily.

Looking forward to this one - I really like how this one will help us in the future to make both - our PRs and installation mechanisms we tell our users, to make them resilient to similar accidents broken packages released on PyPI. I really look forward to this one to make it part of our regular process. The existing workarounds are complicated and non-end-user friendly at all and would complicate our CI process quite a lot as we woudl have to combine regular and build constraints dynamically on our CI.

notatallshaw · 2025-08-11T13:47:41Z

Thanks @potiuk, a few things worth nothing though:

This only allows users to protect themselves from breakages, not libraries. I feel like there is an open question about whether libraries (or popular applications) should provide upperbounds on their build dependencies. In fact it was recently discussed in pypa/packaging.python.org#1880 (comment)

This doesn't technically introduce any new functionality if you have full control of pip you can reproduce all of this:

Install only constraints: --constraint
Install and build time constraints single file: PIP_CONSTRAINT
Install and build time constraints separate files: --constraint and PIP_CONSTRAINT
Build time only constraints: --constraint pointed to an empty file and PIP_CONSTRAINT

But this is poor UX, it means you don't have the usual CLI and ENV flexibility, and the fact this works the way it does depends on multiple internal details of how pip is implemented. In fact what finally inspired me to write this PR is that one of those internal details (installing build dependencies via calling pip in a subprocess) is likely to change soon.

potiuk · 2025-08-11T15:50:09Z

Yep. I agree 100% with everythi g you wrote. I already started lookng at implementing a co bo of co ateaints solution in our CI this morning - bit I was so relieved I did not have to (yet) as today's aetuptools-scm 9.1.0 was yanked - precisely because of the poor UX and the undocumented, internal behaviour and potential conflicts between build and regular constraints which has kinda unspecified behaviour.

And - as for Airflow - we simply pin our build dependencies - because in our case reproducibility trumps potential security issues with build dependencies - and also it makes it easier for us because we are pure Python only so very few upstream users would actually use .sdist to build airflow - and if they do, they usually do it in controlled environments.

ichard26

This looks quite good already! Thanks a lot for thinking through the scenarios and interactions thoroughly. I'm impressed with your unit and functional test.

This is my first review pass which mostly consists of minor comments. I'll want to do another pass before I approve.

A general note is that the codebase seems to have standardized around constraints for --constraint while this PR uses build_constraint for --build-constraints. I'd prefer that we use build_constraints with a S internally for consistency, but I do acknowledge that constraints is not consistent with the CLI.

ichard26 · 2025-08-13T21:50:16Z

src/pip/_internal/build_env.py

+        if not self._constraints:
+            return
+
+        if not os.environ.get("PIP_CONSTRAINT"):
+            return
+
+        pip_constraint_files = [
+            f for f in os.environ["PIP_CONSTRAINT"].split() if f.strip()
+        ]
+        if pip_constraint_files and pip_constraint_files == self._constraints:


I assume the scenario where the stored constraints do not match PIP_CONSTRAINT mostly involve configuration files?

This function got a little confused because I was initially going to throw the deprecation in the isolated build process, but given that doesn't output to the console by default that would have been a poor place to put it.

On reconsideration we should emit a deprecation if PIP_CONSTRAINT is at all non-empty. I will update.

configuration files?

I was mostly hoping that users aren't using configuration files to have build processes be constrained. I don't really know how to detect that.

Updated, check is far more simple now.

ichard26 · 2025-08-13T21:51:41Z

src/pip/_internal/build_env.py

@@ -112,6 +164,8 @@ def install(
        kind: str,
        for_req: InstallRequirement | None,
    ) -> None:
+        self._deprecation_constraint_check()


We should probably issue this deprecation warning only once. In some scenarios, even a single build will trigger this deprecation twice as the build backend can request additional dependencies dynamically.

... although I say that having tried this locally... I can't get the deprecation to be printed twice. Hmm.

Updated, I'm not so sure this prevents it being called in the isolated build sub-process, but it will only be called once per process at least.

ichard26 · 2025-08-13T21:54:14Z

src/pip/_internal/build_env.py

+class ExtraEnviron(TypedDict, total=False):
+    extra_environ: dict[str, str]


What's the benefit of defining this typed dictionary? It seems superfluous IMO.

I am all open to suggestions that don't make the code more awkward (e.g writing a much larger if block) and keep mypy happy.

I could move it into the if type checking block though to eliminate any run time construction.

I've moved in to type checking block, not sure what else to do.

ichard26 · 2025-08-13T21:55:22Z

src/pip/_internal/cli/cmdoptions.py

+def check_build_constraints(options: Values) -> None:
+    """Function for validating build constraint options.


Good forward thinking here!

tests/functional/test_build_constraints.py

src/pip/_internal/cli/req_command.py

news/13534.feature.rst

src/pip/_internal/build_env.py

Co-authored-by: Richard Si <[email protected]>

…ints

…precation_constraint_check`

…sent

notatallshaw · 2025-08-21T01:18:50Z

I believe I have addressed all of @ichard26's review items as best as I can.

notatallshaw added 4 commits August 8, 2025 22:39

Implement build constraints

2e4086e

Add build constraints tests

1aa1d32

Add build constraints to user guide

db4fcf7

NEWS ENTRY

d9d5f5d

notatallshaw force-pushed the add-build-constraints branch from 60df4a0 to d9d5f5d Compare August 9, 2025 02:46

psf-chronographer bot added the bot:chronographer:provided label Aug 9, 2025

ichard26 self-requested a review August 9, 2025 02:59

pradyunsg reviewed Aug 9, 2025

View reviewed changes

src/pip/_internal/build_env.py Outdated Show resolved Hide resolved

Imply using new behavior when build constraints are provided without …

8d170b5

…needing to use the `build-constraint` feature

notatallshaw force-pushed the add-build-constraints branch from 6c44d43 to 8d170b5 Compare August 9, 2025 14:09

potiuk mentioned this pull request Aug 11, 2025

Setuptools 9.1.0 break xmlsec and likely other installation (new issue - not the same as in 9.0.*) pypa/setuptools-scm#1194

Closed

potiuk mentioned this pull request Aug 12, 2025

Add support to --build-constraint(s) flag for our constraint preparation apache/airflow#54394

Open

ichard26 added this to the 25.3 milestone Aug 13, 2025

ichard26 requested changes Aug 13, 2025

View reviewed changes

notatallshaw and others added 3 commits August 13, 2025 20:09

Update src/pip/_internal/cli/req_command.py

9f9032c

Co-authored-by: Richard Si <[email protected]>

Update src/pip/_internal/build_env.py

a9e81d7

Co-authored-by: Richard Si <[email protected]>

Merge branch 'main' into add-build-constraints

4fbafeb

ichard26 mentioned this pull request Aug 14, 2025

Release 25.3 (October) #13522

Open

notatallshaw added 8 commits August 19, 2025 18:54

Fix linting

8943172

Fix test

ef06010

Consistently use "build constraints" in variables and documentation

d564457

Simplify deprecation warning

ebd55e7

Only emit pip constraint deprecation warning once

c41496e

Move ExtraEnviron into type checking block

e015f3d

Use standard assert_installed in functional tests for build constra…

b333b85

…ints

Eagerly assert build constraints files

bc48f0b

notatallshaw added 4 commits August 19, 2025 20:38

Add deprecation news item.

fc1bfb5

Remove pointless check for _PIP_IN_BUILD_IGNORE_CONSTRAINTS in `_de…

74b08e1

…precation_constraint_check`

Exit _deprecation_constraint_check early when build constraints pre…

41164aa

…sent

Remove superfluous constraints parameter

e53db93

		class ExtraEnviron(TypedDict, total=False):
		extra_environ: dict[str, str]

		def check_build_constraints(options: Values) -> None:
		"""Function for validating build constraint options.

Add build constraints #13534

Are you sure you want to change the base?

Add build constraints #13534

Uh oh!

Conversation

notatallshaw commented Aug 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

potiuk commented Aug 11, 2025

Uh oh!

potiuk commented Aug 11, 2025

Uh oh!

notatallshaw commented Aug 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

potiuk commented Aug 11, 2025

Uh oh!

ichard26 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

notatallshaw commented Aug 21, 2025

Uh oh!

Uh oh!

notatallshaw commented Aug 9, 2025 •

edited

Loading

notatallshaw commented Aug 11, 2025 •

edited

Loading