-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Add incompatibility from proxy to base package #15200
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
a289813
to
1774762
Compare
1774762
to
18e913d
Compare
18e913d
to
62a47ce
Compare
Tagging both zanie and andrew because in an older version I broke a conflict test by accidentally including dependency groups, we should ensure that there are no other gotchas with dependency groups and/or conflicts. |
5b498fb
to
e434f41
Compare
e434f41
to
dd2fb4a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool. I do not have a deep understanding of the implications here — it makes sense to have another reviewer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this makes sense to me! This is a nice improvement. :-)
----- stderr ----- | ||
× No solution found when resolving dependencies: | ||
╰─▶ Because only datasets<=2.18.0 is available and your project depends on datasets>=2.19, we can conclude that your project's requirements are unsatisfiable. | ||
╰─▶ Because your project depends on datasets<2.19 and datasets>=2.19, we can conclude that your project's requirements are unsatisfiable. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh nice!
and pandas==1.5.3 depends on numpy{python_full_version >= '3.10'}>=1.21.0, we can conclude that pandas==1.5.3 depends on numpy>=1.21.0. | ||
And because your project depends on numpy==1.20.3 and pandas==1.5.3, we can conclude that your project's requirements are unsatisfiable. | ||
╰─▶ Because pandas==1.5.3 depends on numpy{python_full_version >= '3.10'}>=1.21.0 and your project depends on numpy==1.20.3, we can conclude that your project and pandas==1.5.3 are incompatible. | ||
And because your project depends on pandas==1.5.3, we can conclude that your project's requirements are unsatisfiable. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Much better!
Attempt at #15199 by adding an incompatibility that lets pubgrub skip of marker packages when the base package already has an incompatible version. There are some false positives we still need to figure out, specifically around conflicts. The change looks perf neutral for the one case I tried: ``` $ hyperfine --warmup 2 "target/release/uv pip compile --universal scripts/requirements/airflow.in" "target/release/uv pip compile --universal scripts/requirements/airflow.in" Benchmark 1: target/release/uv pip compile --universal scripts/requirements/airflow.in Time (mean ± σ): 482.5 ms ± 12.8 ms [User: 602.7 ms, System: 205.2 ms] Range (min … max): 468.7 ms … 503.2 ms 10 runs Benchmark 2: target/release/uv pip compile --universal scripts/requirements/airflow.in Time (mean ± σ): 479.3 ms ± 21.0 ms [User: 604.0 ms, System: 200.5 ms] Range (min … max): 464.1 ms … 537.5 ms 10 runs Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options. Summary target/release/uv pip compile --universal scripts/requirements/airflow.in ran 1.01 ± 0.05 times faster than target/release/uv pip compile --universal scripts/requirements/airflow.in ```
dd2fb4a
to
a894509
Compare
No changes in any tested package. |
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.8.17` -> `0.8.22` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.8.22`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0822) [Compare Source](astral-sh/uv@0.8.21...0.8.22) Released on 2025-09-23. ##### Python - Upgrade Pyodide to 0.28.3 ([#​15999](astral-sh/uv#15999)) ##### Security - Upgrade `astral-tokio-tar` to 0.5.5 which [hardens tar archive extraction](GHSA-3wgq-wrwc-vqmv) ([#​16004](astral-sh/uv#16004)) ### [`v0.8.21`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0821) [Compare Source](astral-sh/uv@0.8.20...0.8.21) Released on 2025-09-23. ##### Enhancements - Refresh lockfile when `--refresh` is provided ([#​15994](astral-sh/uv#15994)) ##### Preview features Add support for S3 request signing ([#​15925](astral-sh/uv#15925)) ### [`v0.8.20`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0820) [Compare Source](astral-sh/uv@0.8.19...0.8.20) Released on 2025-09-22. ##### Enhancements - Add `--force` flag for `uv cache clean` ([#​15992](astral-sh/uv#15992)) - Improve resolution errors with proxied packages ([#​15200](astral-sh/uv#15200)) ##### Preview features - Allow upgrading pre-release versions of the same minor Python version ([#​15959](astral-sh/uv#15959)) ##### Bug fixes - Hide `freethreaded+debug` Python downloads in `uv python list` ([#​15985](astral-sh/uv#15985)) - Retain the cache lock and temporary caches during `uv run` and `uvx` ([#​15990](astral-sh/uv#15990)) ##### Documentation - Add `package` level conflicts to the conflicting dependencies docs ([#​15963](astral-sh/uv#15963)) - Document pyodide support ([#​15962](astral-sh/uv#15962)) - Document support for free-threaded and debug Python versions ([#​15961](astral-sh/uv#15961)) - Expand the contribution docs on issue selection ([#​15966](astral-sh/uv#15966)) - Tweak title for viewing version in project guide ([#​15964](astral-sh/uv#15964)) ### [`v0.8.19`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0819) [Compare Source](astral-sh/uv@0.8.18...0.8.19) Released on 2025-09-19. ##### Python - Add CPython 3.14.0rc3 - Upgrade OpenSSL to 3.5.3 See the [python-build-standalone release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250918) for more details. ##### Bug fixes - Make `uv cache clean` parallel process safe ([#​15888](astral-sh/uv#15888)) - Fix implied `platform_machine` marker for `win_arm64` platform tag ([#​15921](astral-sh/uv#15921)) ### [`v0.8.18`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0818) [Compare Source](astral-sh/uv@0.8.17...0.8.18) Released on 2025-09-17. ##### Enhancements - Add PyG packages to torch backend ([#​15911](astral-sh/uv#15911)) - Add handling for unnamed conda environments in base environment detection ([#​15681](astral-sh/uv#15681)) - Allow selection of debug build interpreters ([#​11520](astral-sh/uv#11520)) - Improve `uv init` defaults for native build backend cache keys ([#​15705](astral-sh/uv#15705)) - Error when `pyproject.toml` target does not exist for dependency groups ([#​15831](astral-sh/uv#15831)) - Infer check URL from publish URL when known ([#​15886](astral-sh/uv#15886)) - Support Gitlab CI/CD as a trusted publisher ([#​15583](astral-sh/uv#15583)) - Add GraalPy 25.0.0 with support for Python 3.12 ([#​15900](astral-sh/uv#15900)) - Add `--no-clear` to `uv venv` to disable removal prompts ([#​15795](astral-sh/uv#15795)) - Add conflict detection between `--only-group` and `--extra` flags ([#​15788](astral-sh/uv#15788)) - Allow `[project]` to be missing from a `pyproject.toml` ([#​14113](astral-sh/uv#14113)) - Always treat conda environments named `base` and `root` as base environments ([#​15682](astral-sh/uv#15682)) - Improve log message when direct build for `uv_build` is skipped ([#​15898](astral-sh/uv#15898)) - Log when the cache is disabled ([#​15828](astral-sh/uv#15828)) - Show pyx organization name after authenticating ([#​15823](astral-sh/uv#15823)) - Use `_CONDA_ROOT` to detect Conda base environments ([#​15680](astral-sh/uv#15680)) - Include blake2b hash in `uv publish` upload form ([#​15794](astral-sh/uv#15794)) - Fix misleading debug message when removing environments in `uv sync` ([#​15881](astral-sh/uv#15881)) ##### Deprecations - Deprecate `tool.uv.dev-dependencies` ([#​15469](astral-sh/uv#15469)) - Revert "feat(ci): build loongarch64 binaries in CI ([#​15387](astral-sh/uv#15387))" ([#​15820](astral-sh/uv#15820)) ##### Preview features - Propagate preview flag to client for `native-auth` feature ([#​15872](astral-sh/uv#15872)) - Store native credentials for realms with the https scheme stripped ([#​15879](astral-sh/uv#15879)) - Use the root index URL when retrieving credentials from the native store ([#​15873](astral-sh/uv#15873)) ##### Bug fixes - Fix `uv sync --no-sources` not switching from editable to registry installations ([#​15234](astral-sh/uv#15234)) - Avoid display of an empty string when a path is the working directory ([#​15897](astral-sh/uv#15897)) - Allow cached environment reuse with `@latest` ([#​15827](astral-sh/uv#15827)) - Allow escaping spaces in --env-file handling ([#​15815](astral-sh/uv#15815)) - Avoid ANSI codes in debug! messages ([#​15843](astral-sh/uv#15843)) - Improve BSD tag construction ([#​15829](astral-sh/uv#15829)) - Include SHA when listing lockfile changes ([#​15817](astral-sh/uv#15817)) - Invert the logic for determining if a path is a base conda environment ([#​15679](astral-sh/uv#15679)) - Load credentials for explicit members when lowering ([#​15844](astral-sh/uv#15844)) - Re-add `triton` as a torch backend package ([#​15910](astral-sh/uv#15910)) - Respect `UV_INSECURE_NO_ZIP_VALIDATION=1` in duplicate header errors ([#​15912](astral-sh/uv#15912)) ##### Documentation - Add GitHub Actions to PyPI trusted publishing example ([#​15753](astral-sh/uv#15753)) - Add Coiled integration documentation ([#​14430](astral-sh/uv#14430)) - Add verbose output to the getting help section ([#​15915](astral-sh/uv#15915)) - Document `NO_PROXY` support ([#​15816](astral-sh/uv#15816)) - Document cache-keys for native build backends ([#​15811](astral-sh/uv#15811)) - Add documentation for dependency group `requires-python` ([#​14282](astral-sh/uv#14282)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xMTUuNiIsInVwZGF0ZWRJblZlciI6IjQxLjEyNS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Add an incompatibility that lets pubgrub skip of marker packages when the base package already has an incompatible version to improve the error messages (#15199).
The change is also a small perf improvement. Overall this should be able to improve performance in slow cases by avoiding trying proxy package versions that are impossible anyway, for a (ideally very small cost) for tracking the additional incompatibility and tracking the base package for each proxy package.
This implementation is the basic version: When we see a proxy
foo{...}>=x,<y
we add a dependency edgefoo{...}>=x,<y
->foo>=x,<y
. There are several way to extend this, which likely help more with performance than with error messages.One idea is that if we see
foo{...}>=x,<y
but we already made a selection forfoo==z
outside that range, we can insert a dependencyfoo{...}!=z
->foo!=z
. This avoids trying any version of the proxy package except the version that matches our previous selection.Another is that if we see a dependency
foo>=x,<y
, we also addfoo{...}>=x,y
->foo>=x,<y
. This allows backtracking beyondfoo
immediately if all version offoo{...}>=x,<y
are incompatible, sincefoo{...}>=x,<y
incompatible ->foo>=x,<y
incompatible -> the package that depended offoo>=x,<y
is incompatible.The cost for each of these operations is tracking an additional incompatibility per virtual package. An alternative approach is to only add the incompatibility lazily, only when we've tried several version of the virtual package already. This needs to be weighed of with the better error messages that the incompatibility gives, we unfortunately have only few large reference examples.
Requires astral-sh/pubgrub#45
Closes #15199