-
Notifications
You must be signed in to change notification settings - Fork 316
Add x-azcopy
and x-azcopy-sas
binary caching sources
#1738
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
x-azcopy
providerx-azcopy
and x-azcopy-sas
binary caching sources
b7f6328
to
1065502
Compare
Back to draft while the |
a8e6ee3
to
3e09823
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only comment I'm 'request changes' over is the one on line 1078 where the for loop seems clearly broken. The other things are nitpicks.
Co-authored-by: Billy O'Neal <[email protected]>
b300bb9
to
ea49e5c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds two new binary caching sources for vcpkg that use Azure Storage blob containers via the azcopy
executable. The implementation provides both SAS token-based authentication and Microsoft Entra ID authentication options.
Key changes:
- Introduces
x-azcopy
andx-azcopy-sas
binary cache providers as alternatives to existing Azure blob storage options - Implements batch downloading using
azcopy copy --include-path
for improved performance compared to individual file downloads - Adds comprehensive test coverage for both new provider types with various configuration scenarios
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
File | Description |
---|---|
src/vcpkg/metrics.cpp | Adds metrics tracking for the two new azcopy binary cache providers |
src/vcpkg/binarycaching.cpp | Core implementation of AzCopy providers with batch download/upload logic and configuration parsing |
src/vcpkg-test/configparser.cpp | Comprehensive unit tests for both azcopy provider variants with various configuration scenarios |
locales/messages.json | Adds localized message for Azure Storage restoration feedback |
include/vcpkg/metrics.h | Defines new metric types for tracking azcopy provider usage |
include/vcpkg/binarycaching.h | Declares AzCopyUrl structure and adds provider state fields |
include/vcpkg/base/message-data.inc.h | Declares the new Azure Storage restoration message |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Adds two new binary sources that interact with Azure Storage blob containers using the
azcopy
executable.x-azcopy-sas,<url>,<sas>[,<rw>]
: Intended as a drop-in replacement forazblob
, it uses SAS tokens for authentication.x-azcopy,<url>[,<rw>]
: Intended for non-SAS authentication methods like Microsoft Entra ID; requires preloading credentials or/and theAZCOPY_AUTO_LOGIN_TYPE
.Internally, we use
azcopy copy --from-to BlobLocal {destination} --include-path {files}
argument to download multiple files in a singleazcopy
job, this improves performance and brings it more in line withazblob
, that uses a singlecurl
invocation to download a batch of files. Upload happens one file at a time in the background.Small test
Large test
Related:
x-azcopy
andx-azcopy-sas
binary cache sources. vcpkg#46705, results: https://dev.azure.com/vcpkg/public/_build/results?buildId=118599&view=resultsx-azcopy
andx-azcopy-sas
binary caching sources MicrosoftDocs/vcpkg-docs#511)