Skip to content

Commit 4bb14c5

Browse files
authored
Merge branch 'net10.0' into fix-21814
2 parents fb8535a + 3b8f78f commit 4bb14c5

File tree

79 files changed

+3105
-520
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

79 files changed

+3105
-520
lines changed

NuGet.config

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,8 @@
2727
<add key="darc-pub-dotnet-maui-a33a875e" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-maui-a33a875e/nuget/v3/index.json" />
2828
<!-- Added manually for dotnet/runtime 8.0.16 -->
2929
<add key="darc-pub-dotnet-runtime-6cd8ef8" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-runtime-6cd8ef87/nuget/v3/index.json" />
30-
<!-- Added manually for dotnet/runtime 9.0.5 -->
31-
<add key="darc-pub-dotnet-runtime-207cda3" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-runtime-207cda34/nuget/v3/index.json" />
30+
<!-- Added manually for dotnet/runtime 9.0.6 -->
31+
<add key="darc-pub-dotnet-runtime-df017025" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-runtime-df017025/nuget/v3/index.json" />
3232
<!-- Added manually for .NET 9 Android -->
3333
<add key="darc-pub-dotnet-android-1719a35" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-android-1719a35b/nuget/v3/index.json" />
3434
<!-- Added manually for .NET 8 Android -->

docs/ReleaseProcess.md

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
# .NET MAUI Release Process Using Arcade
2+
3+
This document describes the .NET MAUI release process, which uses the Arcade SDK for building packages and publishing them to NuGet.org and Workload Set channels.
4+
5+
## Overview
6+
7+
The .NET MAUI release process consists of two main phases:
8+
1. Building and packing using `azure-pipelines-internal.yml`
9+
2. Publishing to NuGet.org and Workload Set channels using `maui-release-internal.yml`
10+
11+
This process leverages the [.NET Arcade infrastructure](https://github.com/dotnet/arcade), which is a set of shared tools and services used across the .NET ecosystem to standardize build processes, dependency management, and package publishing.
12+
13+
## Build and Pack Pipeline (`azure-pipelines-internal.yml`)
14+
15+
The `azure-pipelines-internal.yml` pipeline is responsible for building, packing, and signing the .NET MAUI packages and workloads. This pipeline runs automatically on a schedule (daily at 5:00 UTC) for the main branch and is also triggered on commits to main, release branches, and tags. The pipeline runs in the internal Azure DevOps environment ([dnceng/internal](https://dev.azure.com/dnceng/internal/_git/dotnet-maui)) where it has access to signing certificates and secured resources.
16+
17+
### Key Steps in Build and Pack Pipeline
18+
19+
1. **Source Provisioning**: Sets up the build environment with necessary dependencies like Android SDKs.
20+
21+
2. **Build**: Builds the .NET MAUI projects in the repository.
22+
23+
3. **Pack**: Creates NuGet packages for the .NET MAUI libraries.
24+
25+
4. **Sign**: Signs the packages with Microsoft's certificate (only on official internal builds).
26+
27+
5. **Build Workloads**: Constructs the workload manifests and packages required for the .NET MAUI SDK.
28+
29+
6. **Publish to BAR**: After successful build and pack, the packages' metadata are published to the Build Asset Registry (BAR) in Maestro, which is used by the .NET SDK to consume these packages.
30+
31+
### Key Features
32+
33+
- The pipeline uses the Arcade SDK, which is a shared infrastructure for .NET projects
34+
- All packages are signed using Microsoft's certificate
35+
- Packages are published to internal feeds and registered in the BAR
36+
- BAR IDs assigned to builds are used to track package assets throughout the release process
37+
- Arcade provides dependency management, build orchestration, and signing services
38+
39+
## Release Pipeline (`maui-release-internal.yml`)
40+
41+
The `maui-release-internal.yml` pipeline is responsible for taking the packed artifacts and publishing them to the appropriate channels. This pipeline is not automatically triggered and must be manually run. Like the build pipeline, it also runs in the internal Azure DevOps environment where it has access to the necessary API keys and secured resources.
42+
43+
### Key Steps in Release Pipeline
44+
45+
1. **Publish to Workload Set Channel**:
46+
- Takes the commit hash of the build to be released
47+
- Retrieves the Build Asset Registry (BAR) ID for that commit
48+
- Publishes the workload set to the appropriate .NET SDK workload channel (.NET 8, 9, or 10 Workload Release)
49+
- This allows the .NET SDK to consume the .NET MAUI workloads
50+
51+
2. **Release Packs**:
52+
- Requires manual approval
53+
- Takes the packages (excluding manifest packages) from the build
54+
- Pushes them to NuGet.org with retry logic and quota management
55+
56+
3. **Release Manifests**:
57+
- Requires separate manual approval
58+
- Takes only the manifest packages from the build
59+
- Pushes them to NuGet.org with retry logic and quota management
60+
61+
### Important Parameters
62+
63+
The release pipeline accepts several parameters:
64+
- `commitHash`: The commit hash to download NuGet packages from
65+
- `pushWorkloadSet`: Whether to publish to the Workload Set channel
66+
- `pushNugetOrg`: Whether to push to NuGet.org
67+
- `pushPackages`: Controls if packages are actually pushed (allows for dry runs)
68+
- `nugetIncludeFilters` and `nugetExcludeFilters`: Filters for controlling which packages are published
69+
70+
## Release Flow
71+
72+
The complete release process follows these steps:
73+
74+
1. Build and package using `azure-pipelines-internal.yml`
75+
- This happens automatically on the main branch and release branches
76+
- The build produces NuGet packages and workload manifests
77+
- The build is assigned a BAR ID in Maestro
78+
- All assets are published to internal feeds and registered in the BAR using darc
79+
80+
2. Determine the commit hash of the build to be released
81+
82+
3. Run the `maui-release-internal.yml` pipeline with:
83+
- The commit hash of the build to release
84+
- Parameters to control which stages to run
85+
- Parameters to control which packages to include or exclude
86+
87+
4. The release pipeline uses Darc to:
88+
- Find the BAR ID associated with the specified commit
89+
- Gather all the packages and assets from internal channel
90+
- Publish the workload set to the appropriate .NET SDK workload channel
91+
92+
5. The release pipeline will then:
93+
- Publish the workload set to the appropriate .NET SDK workload channel
94+
- After manual approval, publish the NuGet packages to NuGet.org
95+
- After a separate manual approval, publish the manifest packages to NuGet.org
96+
97+
6. The packages are now available for consumption via:
98+
- NuGet.org for developers directly referencing the packages
99+
- .NET SDK's workload installation for the complete .NET MAUI development experience
100+
101+
## Build Environment
102+
103+
The .NET MAUI release process operates using two repositories:
104+
105+
1. **Public GitHub Repository**: [https://github.com/dotnet/maui](https://github.com/dotnet/maui)
106+
- Contains all source code and is the primary development repository
107+
- Used for public pull requests and non-official builds
108+
- All contributions and community engagement happen here
109+
110+
2. **Internal Azure DevOps Mirror**: [https://dev.azure.com/dnceng/internal/_git/dotnet-maui](https://dev.azure.com/dnceng/internal/_git/dotnet-maui)
111+
- An internal mirror of the GitHub repository
112+
- Used for official builds, signing, and release processes
113+
- Contains the same code but runs in a secured environment with access to signing certificates and internal resources
114+
- The `azure-pipelines-internal.yml` and `maui-release-internal.yml` pipelines run against this mirror
115+
116+
The use of the internal mirror ensures that the signing process and access to internal feeds are properly secured while still maintaining an open-source development model in the public repository. Changes are synchronized from the public repository to the internal mirror, ensuring that the released packages contain the same code that is publicly visible.
117+
118+
## Arcade, Darc, and Maestro
119+
120+
### Arcade Infrastructure
121+
122+
[Arcade](https://github.com/dotnet/arcade) is Microsoft's .NET Core engineering system, providing shared tools, SDK, and services for .NET repository builds. Arcade standardizes:
123+
124+
- Build environments and scripts
125+
- Package versioning
126+
- Dependency management
127+
- Signing and publishing
128+
- CI/CD integration
129+
130+
### Darc: Dependency and Asset Reuse Coordinator
131+
132+
Darc is a tool within the Arcade infrastructure used to manage dependencies and coordinate asset reuse across the .NET ecosystem. In the .NET MAUI release process, Darc plays a critical role:
133+
134+
1. **Dependency Management**: Manages dependencies between .NET MAUI and other .NET repositories.
135+
2. **Build Asset Tracking**: Associates builds with channels and creates records in the Build Asset Registry (BAR).
136+
3. **Package Discovery**: Locates packages produced by specific commits.
137+
4. **Asset Gathering**: Collects packages for publishing from internal feeds.
138+
139+
### Maestro and the Build Asset Registry (BAR)
140+
141+
Maestro is the service that maintains the Build Asset Registry (BAR), which serves as the central database tracking all assets (packages, blobs, etc.) produced by .NET builds. The release process uses Maestro to:
142+
143+
1. **Track Builds**: Each build is assigned a unique BAR ID that catalogs all assets produced by that build.
144+
2. **Channel Association**: Builds are associated with channels like ".NET 8 Workload Release" or ".NET 9 Workload Release".
145+
3. **Asset Discovery**: The release pipeline uses the BAR ID to locate and gather all assets needed for publishing.
146+
147+
### Publishing Process Flow with Darc
148+
149+
The `maui-release-internal.yml` pipeline uses Darc to:
150+
151+
1. **Find the Build**: Using the commit hash parameter, Darc identifies the corresponding build in the BAR.
152+
```powershell
153+
$buildJson = & $darc get-build --ci --repo "${{ parameters.ghRepo }}" --commit "$(COMMIT)" --output-format json
154+
$barId = $buildJson | ConvertFrom-Json | Select-Object -ExpandProperty "id" -First 1
155+
```
156+
157+
2. **Gather Assets**: Once the BAR ID is obtained, Darc collects all the packages from internal feeds.
158+
```powershell
159+
& $darc gather-drop --ci --id $barId -o "$(Build.StagingDirectory)\nupkgs" --azdev-pat $(System.AccessToken) --verbose
160+
```
161+
162+
3. **Channel Association**: For workload sets, Darc adds the build to the appropriate workload channel.
163+
```powershell
164+
& $darc add-build-to-channel --ci --channel "$workloadSetsChannel" --id "$barId" --skip-assets-publishing
165+
```
166+
167+
4. **Publishing**: The gathered packages are then published to NuGet.org and other appropriate channels based on pipeline parameters.
168+
169+
## Notes
170+
171+
- The release process requires appropriate permissions and API keys for the NuGet feeds, these are stored normally on a KeyVault that is associated with the pipeline
172+
- Multiple API keys are used to handle NuGet.org's rate limiting and quota restrictions
173+
- The process includes retry logic to handle transient failures
174+
- Both pipelines run on internal Microsoft infrastructure to ensure security
175+
- All official builds and releases are performed from the internal Azure DevOps mirror repository, not directly from GitHub
176+
- The internal repository is synchronized with the public GitHub repository to ensure released code matches public code
177+
178+
## References
179+
180+
- [Arcade SDK Documentation](https://github.com/dotnet/arcade/blob/main/Documentation/README.md)
181+
- [.NET MAUI Workloads Documentation](https://github.com/dotnet/maui/blob/main/src/Workload/README.md)
182+
- [Darc Documentation](https://github.com/dotnet/arcade/blob/main/Documentation/Darc.md)
183+
- [Maestro and BAR Overview](https://github.com/dotnet/arcade/blob/main/Documentation/Maestro.md)

0 commit comments

Comments
 (0)