-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Optional managed Python archive download cache #12175
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@Gankra could you give this a look too? I'm curious if you have thoughts on the streaming question. @konstin I'm still not sure of the benefits of storing the compressed archives instead of an unpacked one and tweaking the install logic. Immediately unpacking would solve the streaming problem as well as the delayed hash check (which feels awkward here). It'd also help with the automatic install user experience (ref #12122 (comment)) — I think writing a marker file isn't unreasonable but it's harder to reason about when viewing the directories and may be annoying when we start bundling Python distributions into "virtual" environments. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re: stream teeing -- hard to imagine what exactly is up with the borrowchecks without using it in anger but my experiences here are similarly dire frustration so I wouldn't dwell too hard on it, especially since the wins will be marginal compared to how big of a win you've got already.
Could we only have one unpacked tree instead of two unpacked trees, e.g. with (out of band) tracking whether a Python interpreter as installed? I was targeting fixing the test running with this change, so tackling our Python installation behavior in general would change the scope quite a bit. Moving more features such as decompression out of the test path would take test coverage of an important path in the test setup (vs. just the download, which is already a well-tested logic), so I'd prefer to not cache too much outside the case. We could also go the other way and decrease the scope of this PR by only acting on the environment variable that we only use in testing and only then use the two-step split download first, decompress-and-installer later. |
Are you considering one unpacked cache tree and another hard or soft linked tree as two unpacked trees?
I think we'll want at least one test case that does not use this new cache. |
Sounds great, but is that this change or a different PR, i.e. would that behavior change fix the download behavior?
Sounds good. |
I don't recall where we landed on this during the in-person discussion. Do you? :D |
Part of #11834 Split the download-and-extract of manged Python interpreters into a download to the cache and an extract-and-install phase. This allows quickly re-installing Python interpreters when they are cached, and caching Python installation in tests and CI (follow-up PR). Ideally, we would still have a combined download-and-extract step if the interpreter is available locally, but I couldn't figure out how to tee the stream in reasonable complexity. I nearly succeeded going through a futures `Stream`, but I couldn't figure out how to pass the second writer to the `Stream::then` `FnMut` in a way the borrow checker would accept. Locally for me, `cargo test -p uv -- python_install` goes from 43s to 7s when setting `UV_PYTHON_CACHE_DIR`.
dcf2b42
to
6d355c2
Compare
We decided to change this change to be only about cases where |
## Summary The only thing that changed for #12175 relevant to the existing downloads is the order of nesting, so we're checking all nested IO errors instead of only the first one. See #13238 ## Test Plan This is an educated guess based on what happens if I turn off the network during a download. ``` Downloading cpython-3.13.3-linux-x86_64-gnu (download) (20.3MiB) TRACE Considering retry of error: ExtractError("cpython-3.13.3-20250409-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", Io(Custom { kind: Other, error: TarError { desc: "failed to unpack `/home/konsti/.local/share/uv/python/.temp/.tmpe3AIvt/python/lib/libpython3.13.so.1.0`", io: Custom { kind: Other, error: TarError { desc: "failed to unpack `python/lib/libpython3.13.so.1.0` into `/home/konsti/.local/share/uv/python/.temp/.tmpe3AIvt/python/lib/libpython3.13.so.1.0`", io: Custom { kind: Other, error: reqwest::Error { kind: Decode, source: reqwest::Error { kind: Body, source: TimedOut } } } } } } })) TRACE Cannot retry IO error: not one of `ConnectionReset` or `UnexpectedEof` TRACE Cannot retry IO error: not one of `ConnectionReset` or `UnexpectedEof` TRACE Cannot retry error: not an IO error error: Failed to install cpython-3.13.3-linux-x86_64-gnu Caused by: Failed to extract archive: cpython-3.13.3-20250409-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz Caused by: failed to unpack `/home/konsti/.local/share/uv/python/.temp/.tmpe3AIvt/python/lib/libpython3.13.so.1.0` Caused by: failed to unpack `python/lib/libpython3.13.so.1.0` into `/home/konsti/.local/share/uv/python/.temp/.tmpe3AIvt/python/lib/libpython3.13.so.1.0` Caused by: error decoding response body Caused by: request or response body error Caused by: operation timed out ```
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | minor | `0.6.16` -> `0.7.3` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.7.3`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#073) [Compare Source](astral-sh/uv@0.7.2...0.7.3) ##### Enhancements - Add `--dry-run` support to `uv self update` ([#​9829](astral-sh/uv#9829)) - Add `--show-with` to `uv tool list` to list packages included by `--with` ([#​13264](astral-sh/uv#13264)) - De-duplicate fetched index URLs ([#​13205](astral-sh/uv#13205)) - Support more zip compression formats: bzip2, lzma, xz, zstd ([#​13285](astral-sh/uv#13285)) - Add support for downloading GraalPy ([#​13172](astral-sh/uv#13172)) - Improve error message when a virtual environment Python symlink is broken ([#​12168](astral-sh/uv#12168)) - Use `fs_err` for paths in symlinking errors ([#​13303](astral-sh/uv#13303)) - Minify and embed managed Python JSON at compile time ([#​12967](astral-sh/uv#12967)) ##### Preview features - Build backend: Make preview default and add configuration docs ([#​12804](astral-sh/uv#12804)) - Build backend: Allow escaping in globs ([#​13313](astral-sh/uv#13313)) - Build backend: Make builds reproducible across operating systems ([#​13171](astral-sh/uv#13171)) ##### Configuration - Add `python-downloads-json-url` option for `uv.toml` to configure custom Python installations via JSON URL ([#​12974](astral-sh/uv#12974)) ##### Bug fixes - Check nested IO errors for retries ([#​13260](astral-sh/uv#13260)) - Accept `musllinux_1_0` as a valid platform tag ([#​13289](astral-sh/uv#13289)) - Fix discovery of pre-release managed Python versions in range requests ([#​13330](astral-sh/uv#13330)) - Respect locked script preferences in `uv run --with` ([#​13283](astral-sh/uv#13283)) - Retry streaming downloads on broken pipe errors ([#​13281](astral-sh/uv#13281)) - Treat already-installed base environment packages as preferences in `uv run --with` ([#​13284](astral-sh/uv#13284)) - Avoid enumerating sources in errors for path Python requests ([#​13335](astral-sh/uv#13335)) - Avoid re-creating virtual environment with `--no-sync` ([#​13287](astral-sh/uv#13287)) ##### Documentation - Remove outdated description of index strategy ([#​13326](astral-sh/uv#13326)) - Update "Viewing the version" docs ([#​13241](astral-sh/uv#13241)) ### [`v0.7.2`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#072) [Compare Source](astral-sh/uv@0.7.1...0.7.2) ##### Enhancements - Improve trace log for retryable errors ([#​13228](astral-sh/uv#13228)) - Use "error" instead of "warning" for self-update message ([#​13229](astral-sh/uv#13229)) - Error when `uv version` is used with project-specific flags but no project is found ([#​13203](astral-sh/uv#13203)) ##### Bug fixes - Fix incorrect virtual environment invalidation for pre-release Python versions ([#​13234](astral-sh/uv#13234)) - Fix patching of `clang` in managed Python sysconfig ([#​13237](astral-sh/uv#13237)) - Respect `--project` in `uv version` ([#​13230](astral-sh/uv#13230)) ### [`v0.7.1`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#071) [Compare Source](astral-sh/uv@0.7.0...0.7.1) ##### Enhancement - Add support for BLAKE2b-256 ([#​13204](astral-sh/uv#13204)) ##### Bugfix - Revert fix handling of authentication when encountering redirects ([#​13215](astral-sh/uv#13215)) ### [`v0.7.0`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#070) [Compare Source](astral-sh/uv@0.6.17...0.7.0) This release contains various changes that improve correctness and user experience, but could break some workflows; many changes have been marked as breaking out of an abundance of caution. We expect most users to be able to upgrade without making changes. ##### Breaking changes - **Update `uv version` to display and update project versions ([#​12349](astral-sh/uv#12349 Previously, `uv version` displayed uv's version. Now, `uv version` will display or update the project's version. This interface was [heavily requested](astral-sh/uv#6298) and, after much consideration, we decided that transitioning the top-level command was the best option. Here's a brief example: ```console $ uv init example Initialized project `example` at `./example` $ cd example $ uv version example 0.1.0 $ uv version --bump major example 0.1.0 => 1.0.0 $ uv version --short 1.0.0 ``` If used outside of a project, uv will fallback to showing its own version still: ```console $ uv version warning: failed to read project: No `pyproject.toml` found in current directory or any parent directory running `uv self version` for compatibility with old `uv version` command. this fallback will be removed soon, pass `--preview` to make this an error. uv 0.7.0 (4433f41c9 2025-04-29) ``` As described in the warning, `--preview` can be used to error instead: ```console $ uv version --preview error: No `pyproject.toml` found in current directory or any parent directory ``` The previous functionality of `uv version` was moved to `uv self version`. - **Avoid fallback to subsequent indexes on authentication failure ([#​12805](astral-sh/uv#12805 When using the `first-index` strategy (the default), uv will stop searching indexes for a package once it is found on a single index. Previously, uv considered a package as "missing" from an index during authentication failures, such as an HTTP 401 or HTTP 403 (normally, missing packages are represented by an HTTP 404). This behavior was motivated by unusual responses from some package indexes, but reduces the safety of uv's index strategy when authentication fails. Now, uv will consider an authentication failure as a stop-point when searching for a package across indexes. The `index.ignore-error-codes` option can be used to recover the existing behavior, e.g.: ```toml [[tool.uv.index]] name = "pytorch" url = "https://download.pytorch.org/whl/cpu" ignore-error-codes = [401, 403] ``` Since PyTorch's indexes always return a HTTP 403 for missing packages, uv special-cases indexes on the `pytorch.org` domain to ignore that error code by default. - **Require the command in `uvx <name>` to be available in the Python environment ([#​11603](astral-sh/uv#11603 Previously, `uvx` would attempt to execute a command even if it was not provided by a Python package. For example, if we presume `foo` is an empty Python package which provides no command, `uvx foo` would invoke the `foo` command on the `PATH` (if present). Now, uv will error early if the `foo` executable is not provided by the requested Python package. This check is not enforced when `--from` is used, so patterns like `uvx --from foo bash -c "..."` are still valid. uv also still allows `uvx foo` where the `foo` executable is provided by a dependency of `foo` instead of `foo` itself, as this is fairly common for packages which depend on a dedicated package for their command-line interface. - **Use index URL instead of package URL for keyring credential lookups ([#​12651](astral-sh/uv#12651 When determining credentials for querying a package URL, uv previously sent the full URL to the `keyring` command. However, some keyring plugins expect to receive the *index URL* (which is usually a parent of the package URL). Now, uv requests credentials for the index URL instead. This behavior matches `pip`. - **Remove `--version` from subcommands ([#​13108](astral-sh/uv#13108 Previously, uv allowed the `--version` flag on arbitrary subcommands, e.g., `uv run --version`. However, the `--version` flag is useful for other operations since uv is a package manager. Consequently, we've removed the `--version` flag from subcommands — it is only available as `uv --version`. - **Omit Python 3.7 downloads from managed versions ([#​13022](astral-sh/uv#13022 Python 3.7 is EOL and not formally supported by uv; however, Python 3.7 was previously available for download on a subset of platforms. - **Reject non-PEP 751 TOML files in install, compile, and export commands ([#​13120](astral-sh/uv#13120), [#​13119](astral-sh/uv#13119 Previously, uv treated arbitrary `.toml` files passed to commands (e.g., `uv pip install -r foo.toml` or `uv pip compile -o foo.toml`) as `requirements.txt`-formatted files. Now, uv will error instead. If using PEP 751 lockfiles, use the standardized format for custom names instead, e.g., `pylock.foo.toml`. - **Ignore arbitrary Python requests in version files ([#​12909](astral-sh/uv#12909 uv allows arbitrary strings to be used for Python version requests, in which they are treated as an executable name to search for in the `PATH`. However, using this form of request in `.python-version` files is non-standard and conflicts with `pyenv-virtualenv` which writes environment names to `.python-version` files. In this release, uv will now ignore requests that are arbitrary strings when found in `.python-version` files. - **Error on unknown dependency object specifiers ([12811](astral-sh/uv#12811 The `[dependency-groups]` entries can include "object specifiers", e.g. `set-phasers-to = ...` in: ```toml [dependency-groups] foo = ["pyparsing"] bar = [{set-phasers-to = "stun"}] ``` However, the only current spec-compliant object specifier is `include-group`. Previously, uv would ignore unknown object specifiers. Now, uv will error. - **Make `--frozen` and `--no-sources` conflicting options ([#​12671](astral-sh/uv#12671 Using `--no-sources` always requires a new resolution and `--frozen` will always fail when used with it. Now, this conflict is encoded in the CLI options for clarity. - **Treat empty `UV_PYTHON_INSTALL_DIR` and `UV_TOOL_DIR` as unset ([#​12907](astral-sh/uv#12907), [#​12905](astral-sh/uv#12905 Previously, these variables were treated as set to the current working directory when set to an empty string. Now, uv will ignore these variables when empty. This matches uv's behavior for other environment variables which configure directories. ##### Enhancements - Disallow mixing requirements across PyTorch indexes ([#​13179](astral-sh/uv#13179)) - Add optional managed Python archive download cache ([#​12175](astral-sh/uv#12175)) - Add `poetry-core` as a `uv init` build backend option ([#​12781](astral-sh/uv#12781)) - Show tag hints when failing to find a compatible wheel in `pylock.toml` ([#​13136](astral-sh/uv#13136)) - Report Python versions in `pyvenv.cfg` version mismatch ([#​13027](astral-sh/uv#13027)) ##### Bug fixes - Avoid erroring on omitted wheel-only packages in `pylock.toml` ([#​13132](astral-sh/uv#13132)) - Fix display name for `uvx --version` ([#​13109](astral-sh/uv#13109)) - Restore handling of authentication when encountering redirects ([#​13050](astral-sh/uv#13050)) - Respect build options (`--no-binary` et al) in `pylock.toml` ([#​13134](astral-sh/uv#13134)) - Use `upload-time` rather than `upload_time` in `uv.lock` ([#​13176](astral-sh/uv#13176)) ##### Documentation - Changed `fish` completions append `>>` to overwrite `>` ([#​13130](astral-sh/uv#13130)) - Add `pylock.toml` mentions where relevant ([#​13115](astral-sh/uv#13115)) - Add ROCm example to the PyTorch guide ([#​13200](astral-sh/uv#13200)) - Upgrade PyTorch guide to CUDA 12.8 and PyTorch 2.7 ([#​13199](astral-sh/uv#13199)) ### [`v0.6.17`](https://github.com/astral-sh/uv/releases/tag/0.6.17) [Compare Source](astral-sh/uv@0.6.16...0.6.17) #### Release Notes ##### Preview features - Add PyTorch v2.7.0 to GPU backend ([#​13072](astral-sh/uv#13072)) ##### Bug fixes - Avoid panic for invalid Python versions ([#​13077](astral-sh/uv#13077)) - Block scripts from overwriting `python` ([#​13051](astral-sh/uv#13051)) - Check distribution names to handle invalid redirects ([#​12917](astral-sh/uv#12917)) - Check for mismatched package and distribution names on resolver thread ([#​13088](astral-sh/uv#13088)) - Fix panic with invalid last character in PEP 508 name ([#​13105](astral-sh/uv#13105)) - Reject `requires-python` even if not listed on the index page ([#​13086](astral-sh/uv#13086)) #### Install uv 0.6.17 ##### Install prebuilt binaries via shell script ```sh curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.6.17/uv-installer.sh | sh ``` ##### Install prebuilt binaries via powershell script ```sh powershell -ExecutionPolicy Bypass -c "irm https://github.com/astral-sh/uv/releases/download/0.6.17/uv-installer.ps1 | iex" ``` #### Download uv 0.6.17 | File | Platform | Checksum | |--------|----------|----------| | [uv-aarch64-apple-darwin.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-aarch64-apple-darwin.tar.gz) | Apple Silicon macOS | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-aarch64-apple-darwin.tar.gz.sha256) | | [uv-x86\_64-apple-darwin.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-x86\_64-apple-darwin.tar.gz) | Intel macOS | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-x86\_64-apple-darwin.tar.gz.sha256) | | [uv-aarch64-pc-windows-msvc.zip](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-aarch64-pc-windows-msvc.zip) | ARM64 Windows | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-aarch64-pc-windows-msvc.zip.sha256) | | [uv-i686-pc-windows-msvc.zip](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-i686-pc-windows-msvc.zip) | x86 Windows | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-i686-pc-windows-msvc.zip.sha256) | | [uv-x86\_64-pc-windows-msvc.zip](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-x86\_64-pc-windows-msvc.zip) | x64 Windows | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-x86\_64-pc-windows-msvc.zip.sha256) | | [uv-aarch64-unknown-linux-gnu.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-aarch64-unknown-linux-gnu.tar.gz) | ARM64 Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-aarch64-unknown-linux-gnu.tar.gz.sha256) | | [uv-i686-unknown-linux-gnu.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-i686-unknown-linux-gnu.tar.gz) | x86 Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-i686-unknown-linux-gnu.tar.gz.sha256) | | [uv-powerpc64-unknown-linux-gnu.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-powerpc64-unknown-linux-gnu.tar.gz) | PPC64 Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-powerpc64-unknown-linux-gnu.tar.gz.sha256) | | [uv-powerpc64le-unknown-linux-gnu.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-powerpc64le-unknown-linux-gnu.tar.gz) | PPC64LE Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-powerpc64le-unknown-linux-gnu.tar.gz.sha256) | | [uv-s390x-unknown-linux-gnu.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-s390x-unknown-linux-gnu.tar.gz) | S390x Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-s390x-unknown-linux-gnu.tar.gz.sha256) | | [uv-x86\_64-unknown-linux-gnu.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-x86\_64-unknown-linux-gnu.tar.gz) | x64 Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-x86\_64-unknown-linux-gnu.tar.gz.sha256) | | [uv-armv7-unknown-linux-gnueabihf.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-armv7-unknown-linux-gnueabihf.tar.gz) | ARMv7 Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-armv7-unknown-linux-gnueabihf.tar.gz.sha256) | | [uv-aarch64-unknown-linux-musl.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-aarch64-unknown-linux-musl.tar.gz) | ARM64 MUSL Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-aarch64-unknown-linux-musl.tar.gz.sha256) | | [uv-i686-unknown-linux-musl.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-i686-unknown-linux-musl.tar.gz) | x86 MUSL Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-i686-unknown-linux-musl.tar.gz.sha256) | | [uv-x86\_64-unknown-linux-musl.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-x86\_64-unknown-linux-musl.tar.gz) | x64 MUSL Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-x86\_64-unknown-linux-musl.tar.gz.sha256) | | [uv-arm-unknown-linux-musleabihf.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-arm-unknown-linux-musleabihf.tar.gz) | ARMv6 MUSL Linux (Hardfloat) | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-arm-unknown-linux-musleabihf.tar.gz.sha256) | | [uv-armv7-unknown-linux-musleabihf.tar.gz](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-armv7-unknown-linux-musleabihf.tar.gz) | ARMv7 MUSL Linux | [checksum](https://github.com/astral-sh/uv/releases/download/0.6.17/uv-armv7-unknown-linux-musleabihf.tar.gz.sha256) | </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4yNTcuOCIsInVwZGF0ZWRJblZlciI6IjM5LjI2NC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Adds a cache bucket for Python installs and uses it by default during tests, extending the opt-in cache added in #12175 Updates the `python_install` tests to use a shared cache for Python installs. This reduces the `python_install` test runtime on my machine from 23s -> 17s. The difference should be much larger on machines with slower internet and less cores for test workers :) This should also improve stability in CI by reducing reliance on the network during test runs, see #14327
Part of #11834
Currently, all Python installation are a streaming download-and-extract. With this PR, we add the
UV_PYTHON_CACHE_DIR
variable. When set, the installation is split into downloading the interpreter intoUV_PYTHON_CACHE_DIR
and extracting it there from a second step. If the archive is already present inUV_PYTHON_CACHE_DIR
, we skip the download.The feature can be used to speed up tests and CI. Locally for me,
cargo test -p uv -- python_install
goes from 43s to 7s (1,7s in release mode) when settingUV_PYTHON_CACHE_DIR
. It can also be used for offline installation of Python interpreter, by copying the archives to a directory in the offline machine, while the path rewriting is still performed on the target machine on installation.