chore: Update xUnit related package version to v3 #10474
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is Draft PR to migrate unit test projects to
xUnit v3. (Prev draft PR is #10375)Currently test execution on VS is unstable
Known issues
1.TestTfmsInParallel: falsesetting is not works as expected when running tests with VS Test Explorer~~ (
https://github.com/xunit/visualstudio.xunit/issues/429)~~dotnet.configIt need to add
dotnet.configfile to run tests with MTP when using .NET 10 SDK. But currently xUnit.v3 don't support latest version of MTP.-> It'll be removed when .NET 10 RC2 released. and it need to use
global.jsoninstead.What's changed to this PR
1. Switch xUnit.NET related NuGet packages to v3.
Verify.Xunit->Verify.XunitV3xunit->xunit.v32. Add additional output log types (
*.log/*.ctrf)3. Change unittest's projects
OutputType fromLibrarytoExeand Exolicitly setIsTestProject` property4. Add
Microsoft.Testing.Platformrelated settingsMicrosoft.Testing.Platform is alternative unit test execution layers.
6. Add
xunit.runner.jsonconfig file for test project.7. Pass
TestContext.Current.CancellationTokenthat require Cancellation token.8. Modify
MetadataCommandTest.csAnd add steps to create dummyDirectory.Build.propsfile.It's required to ignore project directories
Directory.Build.propssettings9. Add empty
Directory.Build.propsfile to samples directory10. Modify CI workflow to use custom
seedvalue to reproduce issue on CIBy default. xUnit determine
seedon build time and seed value is different between assemblies.So to fix seed value. Use
github.run_numberas a seed value. (It's incremented from 1)If CI failed by parallel execution related issue.
It can re-run job with using same seed.
11. Add
SetBranchNameAttributethat derived fromBeforeAfterTestAttributeWhen running
SeedMarkdowntests before other snapshot tests.It cause snapshot diffs because
GitUtility.cscaching actual branch name.To simplify environment variable set/restore.
Add custom
BeforeAfterTestAttributefor this purpose.12. Add
[assembly:CaptureConsole]attribute to test assemblies.13. Add ProcessHelper class to invoke external command (e.g. dotnet, docfx).
When using
Process.Startto launch external command that stdout.Running tests is not completed when running tests on Test Explorer.
So I've added utility code to ignoring stdout/stdout.
15. Modify
dotnet test --filter Stage=Percycommand. to use-- --filter-trait "Stage=Percy"When
TestingPlatformDotnetTestSupportis enabled.dotnet testcommand's--filterargument is silently ignored. (Same behavior as that reported athttps://github.com/microsoft/testfx/issues/4401)It need modify command to pass additional argument(
-- --filter-trait "Stage=Percy")16. Separate
Chromium Headless Shellinstallation from test stepsChromium is installed on when running
docfx pdfcommand first time.It take times and it affects test execution times.
So creating separate chromium install step that is executed before running
dotnet test.https://github.com/dotnet/docfx/pull/10474/files#diff-b803fcb7f17ed9235f1e5cb1fcd2f5d3b2838429d4368ae4c57ce4436577f03fR43-R48
17. Add
-nodereuse:falseand--no-dependenciesparameter todotnet buildcommand inside testsAfter migrated to xUnit v3.
SamplesTest::Extensionsdotnet buildcommand takes extra 15 minutes on Windows CI.It seems relating to
https://github.com/dotnet/sdk/issues/9452so I've added-nodereuse:falseparameter to resolve issue.And adding
--no-dependenciesparameter to suppressDocfx.Appproject build during test.18. Add
dotnet-coveragecommand related settings.Currently Microsoft.Testing.Platform don't support coveret-based codecoverage with
dotnet testcommand.19. Modify custom Fact constructors to suppress xUnit3003 warnings.