Skip to content

Conversation

ahal
Copy link
Collaborator

@ahal ahal commented Sep 5, 2024

No description provided.

@ahal ahal self-assigned this Sep 5, 2024
@ahal ahal marked this pull request as draft September 5, 2024 17:52
@ahal ahal force-pushed the uv branch 8 times, most recently from 30182b3 to 962d6a0 Compare September 6, 2024 13:22
@ahal ahal marked this pull request as ready for review September 6, 2024 13:29
@ahal ahal requested review from a team and jcristau September 6, 2024 13:29
This results in a faster installation over vanilla pip.
@@ -1011,7 +1011,11 @@ def install_pip_requirements(repositories):
return

# TODO: Stop using system Python (#381)
cmd = [sys.executable, "-mpip", "install", "--break-system-packages"]
if shutil.which("uv"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like it could be risky, but I see that taskgraph's own tests are already running OK and end up here, so that's some level of confidence. (I suspect that we may see issues elsewhere for unpredictable reasons, but at least those ought to be caught in PRs as we start adding uv to various docker images and/or workers.)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I'm not sure I'm understanding the risk. Is the concern that uv pip install will differ in some way from pip install? That's a valid concern, but a design goal of uv pip install is to be a drop-in replacement for pip install. So it's certainly possible there are implementation errors, but if it ever deviates from pip install they treat that as a bug. So far my confidence in this tool is very high.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also worth noting that we might not detect this via a patch to the repo if the repo is using an image from DockerHub and not pinning. But that's a tradeoff they decided to make, so I'm not too worried about it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a huge concern or anything, I'm just calling out that changing the way we install packages is an inherent risk and notable change, and that we should watch out for bustage when we start hitting this code path in other projects. Certainly not a blocker for doing this!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying! I do agree there is a level of risk here, but the perf boost is quite substantial. So I think it'll be worthwhile

@ahal ahal merged commit a8508b9 into taskcluster:main Sep 12, 2024
14 checks passed
@ahal ahal deleted the uv branch September 12, 2024 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants