Use `.rcdata` to store trampoline type + path to python binary #15068

paveldikov · 2025-08-04T22:42:08Z

.rsrc is the idiomatic way of storing metadata and non-code resources in PE
binaries. This should make the resulting binaries more robust as they are no longer
dependent on the exact location of a certain magic number.

Addresses: #15022

Test Plan

Existing integration test for uv-trampoline-builder + addition to ensure robustness
to code signing.

crates/uv-trampoline/src/bounce.rs

zanieb · 2025-08-04T23:11:06Z

I'm not super qualified to review this idea, cc @samypr100 & @konstin

T-256

LGTM, it's not necessary to done before merging, but can you confirm that it can bring back python3.7 support? (i.e. uvx -p 3.7 ruff most work)

crates/uv-trampoline-builder/src/lib.rs

crates/uv-trampoline/src/bounce.rs

crates/uv-trampoline/Cargo.toml

paveldikov · 2025-08-05T01:56:27Z

Added an even sloppier (but working) refactor of uv-trampoline-builder. Needs oodles of polish and review.

It is now verifiably robust to code-signing, so that solves my original problem.

I have not verified the py3.7 case, though a similar integration test can be added probably very easily? @T-256

Also note that the Win32 APIs are all really rather unsafe. I'm not sure what the right thing to do here is tbh.

crates/uv-trampoline/src/bounce.rs

crates/uv-trampoline-builder/src/lib.rs

samypr100 · 2025-08-05T02:18:33Z

I don't see any immediate issues with using rcdata for the trampoline kind and runnable path, it's mainly a different way to accomplish the same thing. What I'm not very sure of is using it for the script contents. From a code signing perspective is the goal to sign the launcher regardless of what it may end up executing?

paveldikov · 2025-08-05T10:53:05Z

What I'm not very sure of is using it for the script contents. From a code signing perspective is the goal to sign the launcher regardless of what it may end up executing?

The goal is to be able to sign a (finished) entrypoint executable.

paveldikov · 2025-08-05T11:49:09Z

Getting there...

uv-trampoline is pretty much ready for full review
- Still need to compile binaries for all platforms, not just x86-64 (is this something I need to do by hand? or there's a job somewhere that compiles 'blessed' copies of these?)
uv-trampoline-builder is refactored, but still has a bunch of outstanding things (I might need help with some of them)
- Figure out the TODO'd code path with is_gui (smells wrong, but I couldn't figure out how the previous version avoided having to know this. Does it end up being a don't-care value? It's only used in one other place -- in uv run.)
- There's a dozen warnings that need sorted
- The integration test seems to be flaky on my personal PC. There's a 1 in 3 chance of success, ENOENT or failure to try_from_path(). Not sure why...
- For some reason the PowerShell snippet complains about missing Cert: drive when run on CI

If anyone is able to give me a hand on any of these, it would be amazing! If not I will try to keep chipping away over the next few days

konstin · 2025-08-05T13:47:49Z

Still need to compile binaries for all platforms, not just x86-64 (is this something I need to do by hand? or there's a job somewhere that compiles 'blessed' copies of these?)

You can build the binaries on your machine for testing and CI (there are cross-compile instructions in the readme), before merging the PR a maintainer will build the binaries on their machine and push them to this branch.

crates/uv-trampoline-builder/src/lib.rs

crates/uv-trampoline-builder/Cargo.toml

paveldikov · 2025-08-06T22:20:06Z

Close...

Committed binaries for x86-64 and i686
...I can't seem to get it to x-compile for aarch64 though, hence the failing CI run for aarch64.
A bunch of unsafe Win32 API usage.
- Question to answer: I could try my hand at refactoring it to use the winsafe crate, but current precedent in the codebase seems to steer me away from that. What little Win32 API usage exists right now is unsafe too. Opinions welcome!
Code-signing int test is pretty solid now
Tests passing on Windows! (almost -- 3 test cases failed with what looks like an unrelated transient.)
Tests passing on non-Windows.
- fixed compile on non-Windows. Some serious cfg hole-poking which is not pretty, but it wasn't pretty to begin with, so it's consistent. Some fundamental limitations in cargo workspaces to contend with.
write_to_file() is_gui mystery solved

zanieb · 2025-08-28T13:39:23Z

crates/uv-python/src/managed.rs

-    if cfg!(unix) {
+    #[cfg(unix)]


Note the previous use of if cfg! instead of #[cfg] was intentional so you can edit and check this code when not on a matching platform.

(I'm cleaning this up)

Here's my commit f2b69d2 — I can't push to your repository.

Very weird -- I've ticked the 'accept pushes from maintainers' option. (I have invited you as a collaborator to my fork anyway)

zanieb · 2025-08-28T13:58:06Z

crates/uv/src/commands/project/run.rs

-    let mut file = fs_err::OpenOptions::new()
-        .create_new(true)
-        .write(true)
-        .open(target)?;


Why are these dropped? These are important?

Change of semantics. write_to_file() now takes a file path, not a file handle, and therefore takes care of file creation.

We need it to fail if it's not creating a new file, that's why the original API took a handle.

A little more context at #14790 (comment)

Thanks, I wasn't aware of this requirement. (I still don't get it, but I trust that it's valid)

IIRC I refactored it this way in order to avoid double-opening the file, which is fraught on Windows. Using a fs_err::write_file() one-and-done is nice because it leaves no contention for write_resources() which likewise opens the file, but in a winapi way. (none of the winapi methods accept native Rust File objects, so you couldn't even properly re-use the file handle either.)

Maybe add a boolean exists_ok arg to write_to_file()? And replace fs_err::write_file() with a deliberately scoped block with the appropriate OpenOptions... maybe that will do the trick?

I will look a little closer

I think the most immediate solution is to include an API which performs the same two-step operation as windows_script_launcher where it writes everything to a temporary file, reads that to get the bytes, then writes that to an open file handle.

As a separate note, the two step operation in windows_script_launcher is pretty weird for normal operation! We're writing everything just to read it then write it again :) I worry about the performance implications of that and would like to refactor to avoid that unless the caller needs ownership of the file handle (e.g., as above).

Unfortunately, the general pattern of using write_file then write_resources does leave room for races (and this is not just theoretical, things like this have bit us before). I think it's okay for now, but we'll probably want to look for a way to avoid that too.

Circling back, I guess the two step approach of windows_script_launcher actually does resolve this problem... which hints to a simple solution: just construct the launcher in a temporary file that we're the sole writer to, then rename it into place (this is what we do for atomic writes in general, e.g., see write_atomic).

Happy for you to decide which way you want to go? I do think that the two-step operation is not optimal (or at least not aesthetically pleasing), although I kept it as such to avoid gratuitous API churn -- it's a chunky PR already. But if it does incidentally give us safety, then I guess that's alright. Although I imagine it is better to make the atomicity explicit rather than incidental!

Yeah it looks like there'd be a bunch of API churn here because we need to hash the contents for the RECORD when writing entrypoints.

I think we should change it. I'm vaguely worried about this doubling the overhead of writing entrypoints on Windows, but they're small so I can live with it for now. It makes sense to do separately.

d4303fd addresses my blocking comment, I think.

…path to python binary `.rsrc` is the idiomatic way of storing metadata and non-code resources in PE binaries. This should make the resulting binaries more robust as they are no longer dependent on the exact location of a certain magic number. Addresses: astral-sh#15022

Addresses: astral-sh#15022

T-256 · 2025-08-29T00:16:19Z

FWIW before merge, I just tested binary of x86_64 got from ci summary with Python3.7.
Unfortunately, using zip payload in a section of PE doesn't help with Python3.7's zipimport module.
To keep compatible with Py3.7, I'd recommend to add zip (script_data) to end of target file instead of save it on a section of PE format. it could be thought as follow-up after merge, too.

paveldikov · 2025-08-29T00:29:21Z

@T-256

To keep compatible with Py3.7, I'd recommend to add zip (script_data) to end of target file instead of save it on a section of PE format.

But then it wouldn't work with code-signing, because fixed positioning of the zip blob gets wrecked by code signing tools.

Also -- the current trampoline is already incompatible with 3.7. (Do we actively want to reintroduce compatibility?)

zanieb · 2025-08-29T12:20:27Z

Also -- the current trampoline is already incompatible with 3.7. (Do we actively want to reintroduce compatibility?)

No — if this introduced compatibility that'd be a nice property, but it's not an active goal for the project.

paveldikov · 2025-10-13T13:41:06Z

Hello! Sorry for not engaging much on this PR in the last month and a bit. I am still interested in it -- is there any way I can help?

zanieb · 2025-10-13T13:48:18Z

Sorry I lost track of this too. Let me take a look again!

zanieb · 2025-11-09T04:01:32Z

Cargo.toml

 hyper-util = { version = "0.1.8", features = ["tokio"] }
 ignore = { version = "0.4.23" }
 insta = { version = "1.40.0", features = ["json", "filters", "redactions"] }
+p12 = { version = "0.6.3" }


I wonder if we should be using https://docs.rs/pkcs12/latest/pkcs12/ instead which is maintained?

(It actually looks like that doesn't support writing archives)

Was there a reason P12 format was needed in particular?

I'm not sure.

I also briefly looked at https://github.com/ancwrd1/p12-keystore which seems fine too.

It'd be nice to use a dependency that comes from a higher trust source, even if it's just for a test.

Looking at the test and the Set-AuthenticodeSignature docs, I wonder if its possible to avoid P12 entirely by manually instantiating a System.Security.Cryptography.X509Certificates.X509Certificate2 bundle. I believe that would allows us to only rely on rcgen and not p12 (if it's possible). I'll give it a shot.

Success, although it will require use of pwsh rather than powershell because System.Security.Cryptography.X509Certificates.X509Certificate2::CreateFromPemFile is .NET 5+

The CI should have pwsh already so I don't think its a problem and I don't think we truly need to test in legacy powershell in this particular case as we've proven it works with legacy powershell already using pfx. I'll push my changes soon in case you're interested.

See 153d854

nice! Re the 'why' question: I did it because of a (possibly incorrect) inference that pwsh on CI did not have the ability to modify the credential store

samypr100 · 2025-11-09T05:36:32Z

Cargo.toml

 walkdir = { version = "2.5.0" }
 which = { version = "8.0.0", features = ["regex"] }
-windows = { version = "0.59.0", features = ["Win32_Globalization", "Win32_Security", "Win32_System_Console", "Win32_System_Kernel", "Win32_System_Diagnostics_Debug", "Win32_Storage_FileSystem", "Win32_System_Registry", "Win32_System_IO", "Win32_System_Ioctl"] }
+windows = { version = "0.61.0", features = ["std", "Win32_Globalization", "Win32_System_LibraryLoader", "Win32_System_Console", "Win32_System_Kernel", "Win32_System_Diagnostics_Debug", "Win32_Storage_FileSystem", "Win32_Security", "Win32_System_Registry", "Win32_System_IO", "Win32_System_Ioctl"] }


@zanieb I also don't think we need to change the windows version as part of this PR either. Everything should work fine if we keep the previous windows versions intact. That will minimize the changes to the lock file to be just isolated to rcgen.

After f7e549d windows version is now the same as main. It also removed small unrelated changes to windows_exception in this PR as a result of the upgrade.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.9.8` -> `0.9.9` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.9.9`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#099) [Compare Source](astral-sh/uv@0.9.8...0.9.9) Released on 2025-11-12. ##### Deprecations - Deprecate use of `--project` in `uv init` ([#16674](astral-sh/uv#16674)) ##### Enhancements - Add iOS support to Python interpreter discovery ([#16686](astral-sh/uv#16686)) - Reject ambiguously parsed URLs ([#16622](astral-sh/uv#16622)) - Allow explicit values in `uv version --bump` ([#16555](astral-sh/uv#16555)) - Warn on use of managed pre-release Python versions when a stable version is available ([#16619](astral-sh/uv#16619)) - Allow signing trampolines on Windows by using `.rcdata` to store metadata ([#15068](astral-sh/uv#15068)) - Add `--only-emit-workspace` and similar variants to `uv export` ([#16681](astral-sh/uv#16681)) ##### Preview features - Add `uv workspace dir` command ([#16678](astral-sh/uv#16678)) - Add `uv workspace metadata` command ([#16516](astral-sh/uv#16516)) ##### Configuration - Add `UV_NO_DEFAULT_GROUPS` environment variable ([#16645](astral-sh/uv#16645)) ##### Bug fixes - Remove `torch-model-archiver` and `torch-tb-profiler` from PyTorch backend ([#16655](astral-sh/uv#16655)) - Fix Pixi environment detection ([#16585](astral-sh/uv#16585)) ##### Documentation - Fix `CMD` path in FastAPI Dockerfile ([#16701](astral-sh/uv#16701)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

paveldikov mentioned this pull request Aug 4, 2025

uv-trampoline'd entrypoint .exes are not robust to being signed #15022

Closed

paveldikov commented Aug 4, 2025

View reviewed changes

crates/uv-trampoline/src/bounce.rs Show resolved Hide resolved

paveldikov force-pushed the trampoline branch from 9aaa7dd to 1934b98 Compare August 4, 2025 23:06

T-256 reviewed Aug 4, 2025

View reviewed changes

crates/uv-trampoline-builder/src/lib.rs Outdated Show resolved Hide resolved

crates/uv-trampoline/src/bounce.rs Outdated Show resolved Hide resolved

crates/uv-trampoline/src/bounce.rs Outdated Show resolved Hide resolved

crates/uv-trampoline/Cargo.toml Show resolved Hide resolved

paveldikov force-pushed the trampoline branch from 1934b98 to 7e9c0bf Compare August 5, 2025 01:53

zanieb added the no-build Disable building binaries in CI label Aug 5, 2025

paveldikov force-pushed the trampoline branch from 7e9c0bf to 422893e Compare August 5, 2025 01:58

samypr100 reviewed Aug 5, 2025

View reviewed changes

paveldikov force-pushed the trampoline branch from 422893e to 4d19373 Compare August 5, 2025 10:16

paveldikov force-pushed the trampoline branch 2 times, most recently from 81627de to da84e64 Compare August 5, 2025 11:38

paveldikov marked this pull request as ready for review August 5, 2025 11:49

paveldikov force-pushed the trampoline branch 4 times, most recently from a98aeea to b7bab03 Compare August 5, 2025 21:35

samypr100 reviewed Aug 6, 2025

View reviewed changes

crates/uv-trampoline-builder/src/lib.rs Outdated Show resolved Hide resolved

samypr100 reviewed Aug 6, 2025

View reviewed changes

crates/uv-trampoline-builder/src/lib.rs Outdated Show resolved Hide resolved

paveldikov force-pushed the trampoline branch 3 times, most recently from 7716271 to 12bdf11 Compare August 6, 2025 21:08

T-256 reviewed Aug 6, 2025

View reviewed changes

crates/uv-trampoline-builder/Cargo.toml Outdated Show resolved Hide resolved

paveldikov force-pushed the trampoline branch from 12bdf11 to ef79ad1 Compare August 6, 2025 22:05

zanieb reviewed Aug 28, 2025

View reviewed changes

paveldikov and others added 6 commits August 28, 2025 23:04

tmp: aarch64 trampolines

ee7e578

chore(deps): move windows version to workspace

185bc3f

Addresses: astral-sh#15022

chore(deps): regen lockfile post rebase

cc9c2e5

refactor(uv-trampoline,uv-trampoline-builder): address feedback

97a47f7

Review

509165f

paveldikov force-pushed the trampoline branch from b497296 to a760e39 Compare August 28, 2025 22:14

chore(deps): reconcile lockfile

fd2a9ce

paveldikov force-pushed the trampoline branch from a760e39 to fd2a9ce Compare August 28, 2025 22:19

zanieb added 2 commits September 12, 2025 07:33

Merge branch 'main' into trampoline

3279712

Restore handle API

d4303fd

Merge branch 'main' into trampoline

5687577

konstin changed the title ~~refactor(uv-trampoline): use .rcdata to store trampoline type + path to python binary~~ Use .rcdata to store trampoline type + path to python binary Oct 15, 2025

zanieb added 3 commits November 8, 2025 21:09

Merge branch 'main' into trampoline

a5e08c8

Regenerate lockfile

d8f67c5

Regenerate trampolines

d4131f2

zanieb reviewed Nov 9, 2025

View reviewed changes

fix: remove legacy p12 dependency by using modern approach

153d854

samypr100 reviewed Nov 9, 2025

View reviewed changes

fix: avoid updating windows

f7e549d

zanieb merged commit caf49f8 into astral-sh:main Nov 9, 2025
115 checks passed

BrewTestBot mentioned this pull request Nov 12, 2025

uv 0.9.9 Homebrew/homebrew-core#254185

Merged

Use .rcdata to store trampoline type + path to python binary #15068

Use .rcdata to store trampoline type + path to python binary #15068

Uh oh!

Conversation

paveldikov commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Plan

Uh oh!

Uh oh!

zanieb commented Aug 4, 2025

Uh oh!

T-256 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paveldikov commented Aug 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samypr100 commented Aug 5, 2025

Uh oh!

paveldikov commented Aug 5, 2025

Uh oh!

paveldikov commented Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

konstin commented Aug 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paveldikov commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zanieb Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

T-256 commented Aug 29, 2025

Uh oh!

paveldikov commented Aug 29, 2025

Uh oh!

zanieb commented Aug 29, 2025

Uh oh!

paveldikov commented Oct 13, 2025

Uh oh!

zanieb commented Oct 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Use `.rcdata` to store trampoline type + path to python binary #15068

Use `.rcdata` to store trampoline type + path to python binary #15068

paveldikov commented Aug 4, 2025 •

edited

Loading

paveldikov commented Aug 5, 2025 •

edited

Loading

paveldikov commented Aug 6, 2025 •

edited

Loading

zanieb Aug 29, 2025 •

edited

Loading

samypr100 Nov 9, 2025 •

edited

Loading

samypr100 Nov 9, 2025 •

edited

Loading

paveldikov Nov 9, 2025 •

edited

Loading

samypr100 Nov 9, 2025 •

edited

Loading