Skip to content

[testing] Use latest simulator main #30664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jul 17, 2025
Merged

[testing] Use latest simulator main #30664

merged 3 commits into from
Jul 17, 2025

Conversation

rmarinho
Copy link
Member

Description of Change

Try to pick the latest available iOS simulator based on Xcode select and not hardcoded

rmarinho added 3 commits July 16, 2025 17:16
@rmarinho
[tests] Move to use latest for iOSsimulator
d9c1255 
# Conflicts:
#	eng/pipelines/common/ui-tests.yml
@rmarinho
try fix cake hardcode version
c152fe1
@rmarinho
Try remove this check
df90b49 
# Conflicts:
#	src/Controls/tests/TestCases.Shared.Tests/UITest.cs
@Copilot Copilot AI review requested due to automatic review settings July 16, 2025 16:21
@rmarinho rmarinho requested a review from a team as a code owner July 16, 2025 16:21
Copilot
Copilot AI reviewed Jul 16, 2025
View reviewed changes
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates the iOS simulator version from a hardcoded value to dynamically pick the latest available simulator, and adjusts related build and test scripts to use the new default version.

  • Updated default iOS version constant and matching logic in UI tests.
  • Switched iosVersions in multiple pipeline definitions to 'latest'.
  • Aligned common templates and Cake scripts to use 18.4 as the resolved “latest” simulator version.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/Controls/tests/TestCases.Shared.Tests/UITest.cs Updated _defaultiOSVersion, device-matching logic, and assertion
eng/pipelines/ui-tests.yml Changed iosVersions to 'latest'
eng/pipelines/device-tests.yml Changed iosVersions to 'latest'
eng/pipelines/common/ui-tests.yml Bumped the resolved “latest” version to 18.4 in UI test templates
eng/pipelines/common/device-tests-jobs.yml Added mapping for 'latest' to simulator-64 with API 18.4
eng/devices/ios.cake Updated DefaultVersion and DefaultTestDevice constant

src/Controls/tests/TestCases.Shared.Tests/UITest.cs
@@ -241,7 +243,9 @@ but both can happen.
var device = (string?)((AppiumApp)App).Driver.Capabilities.GetCapability("deviceName")
?? throw new InvalidOperationException("deviceName capability is missing or null.");

if (device.Contains(" Xs", StringComparison.OrdinalIgnoreCase) && platformVersion == "18.0")
environmentName = "ios";
Copy link
Preview

Copilot AI Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This environmentName = "ios"; statement is now unconditional and will override any other environment settings. It should be moved inside the if (device.Contains(" Xs" ...) && platformVersion == _defaultiOSVersion) block to preserve correct logic.

Suggested change
environmentName = "ios";

Copilot uses AI. Check for mistakes.

src/Controls/tests/TestCases.Shared.Tests/UITest.cs
@@ -255,7 +259,7 @@ but both can happen.
}
else
{
Assert.Fail($"iOS visual tests should be run on iPhone Xs (iOS 17.2) or iPhone X (iOS 16.4) simulator images, but the current device is '{deviceName}'. Follow the steps on the MAUI UI testing wiki.");
//Assert.Fail($"iOS visual tests should be run on iPhone Xs (iOS {_defaultiOSVersion}) or iPhone X (iOS 16.4) simulator images, but the current device is '{deviceName}' '{platformVersion}'. Follow the steps on the MAUI UI testing wiki.");
Copy link
Preview

Copilot AI Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Instead of commenting out this assertion, update the failure message to reference the new default iOS version and re-enable the Assert.Fail so unsupported simulator configurations are caught during test setup.

Suggested change
//Assert.Fail($"iOS visual tests should be run on iPhone Xs (iOS {_defaultiOSVersion}) or iPhone X (iOS 16.4) simulator images, but the current device is '{deviceName}' '{platformVersion}'. Follow the steps on the MAUI UI testing wiki.");
Assert.Fail($"iOS visual tests should be run on iPhone Xs (iOS {_defaultiOSVersion}) or iPhone X (iOS 16.4) simulator images, but the current device is '{deviceName}' '{platformVersion}'. Follow the steps on the MAUI UI testing wiki.");

Copilot uses AI. Check for mistakes.

eng/devices/ios.cake
const string DefaultVersion = "18.0";
const string DefaultTestDevice = $"ios-simulator-64_{DefaultVersion}";
const string DefaultVersion = "18.4";
const string DefaultTestDevice = $"ios-simulator-64";
Copy link
Preview

Copilot AI Jul 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Dropping the version suffix from DefaultTestDevice may break scripts that expect a versioned identifier. Consider appending DefaultVersion or updating any downstream logic that parses this string.

Suggested change
const string DefaultTestDevice = $"ios-simulator-64";
const string DefaultTestDevice = $"ios-simulator-64-{DefaultVersion}";

Copilot uses AI. Check for mistakes.

@rmarinho rmarinho merged commit 6137fa1 into main Jul 17, 2025
126 of 129 checks passed
@rmarinho rmarinho deleted the use-latest-simulator-main branch July 17, 2025 09:33
@rmarinho
Copy link
Member Author

/backport to inflight/current

@github-actions GitHub Actions
Copy link
Contributor

Started backporting to inflight/current: https://github.com/dotnet/maui/actions/runs/16341556437

@github-actions github-actions bot mentioned this pull request Jul 17, 2025
Merged
rmarinho added a commit that referenced this pull request Jul 18, 2025
@rmarinho
Revert "[testing] Use latest simulator main (#30664)"
479bda0 
This reverts commit 6137fa1.
rmarinho added a commit that referenced this pull request Jul 18, 2025
@rmarinho
Revert "[testing] Use latest simulator main (#30664)" (#30706)
3f5adff 
This reverts commit 6137fa1.
rmarinho added a commit that referenced this pull request Jul 21, 2025
@rmarinho
Reapply "[testing] Use latest simulator main (#30664)" (#30706)
74e1d41 
This reverts commit 3f5adff.

# Conflicts:
#	eng/pipelines/common/device-tests-jobs.yml
#	eng/pipelines/common/ui-tests.yml
#	src/Controls/tests/TestCases.Shared.Tests/UITest.cs
pictos pushed a commit to pictos/maui that referenced this pull request Jul 24, 2025
@rmarinho
Revert "[testing] Use latest simulator main (dotnet#30664)" (dotnet#3…
fd3a316 
…0706)

This reverts commit 6137fa1.
@github-actions github-actions bot locked and limited conversation to collaborators Aug 17, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

Copilot code review Copilot Copilot left review comments

@StephaneDelcroix StephaneDelcroix Awaiting requested review from StephaneDelcroix StephaneDelcroix is a code owner automatically assigned from dotnet/dotnet-maui-reviewers

@jsuarezruiz jsuarezruiz Awaiting requested review from jsuarezruiz jsuarezruiz is a code owner automatically assigned from dotnet/dotnet-maui-reviewers

Assignees
No one assigned
Labels
area-testing Unit tests, device tests testing-flakiness testing-green-ci
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@rmarinho