Html Logger Error when running System.Text.RegularExpressions.Tests

[eerhardt]

Description

When running the System.Text.RegularExpressions.Tests tests in dotnet/runtime, we are seeing an Html Logger error at the end of the test run. I was able to isolate this into a simple repo.

Steps to reproduce

dotnet test the following project:

UnitTest1.cs

using System.Collections.Generic;
using System.Text.RegularExpressions;
using Xunit;

namespace MyTest;

public class UnitTest1
{
    public static IEnumerable<object[]> AllMatches_TestData()
    {
        yield return new object[] { @"(\uFFFE\uFFFF)+", RegexOptions.None, "=====\uFFFE\uFFFF\uFFFE\uFFFF\uFFFE====",
            new (int, int, string)[] { (5, 4, "\uFFFE\uFFFF\uFFFE\uFFFF") } };
    }

    [Theory]
    [MemberData(nameof(AllMatches_TestData))]
    public void AllMatches(string pattern, RegexOptions options, string input, (int, int, string)[] matches)
    {
    }
}

MyTest.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <Nullable>enable</Nullable>
    <RunSettingsFilePath>.runsettings</RunSettingsFilePath>
    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.9.4" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="3.0.2">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

</Project>

.runsettings

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <LoggerRunSettings>
    <Loggers>
      <Logger friendlyName="html" />
      <Logger friendlyName="console">
        <Configuration>
          <Verbosity>Minimal</Verbosity>
        </Configuration>
      </Logger>
    </Loggers>
  </LoggerRunSettings>
</RunSettings>

Expected behavior

No errors should be logged

Actual behavior

Test run for /workspaces/DotNetTest/MyTest/bin/Debug/net6.0/MyTest.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.0.0-preview-20210817-02
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Html Logger Error : '�', hexadecimal value 0xFFFE, is an invalid character. Line 1, position 448.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: < 1 ms - /workspaces/DotNetTest/MyTest/bin/Debug/net6.0/MyTest.dll (net6.0)

Diagnostic logs

It looks like the exception is coming from around here

vstest/src/Microsoft.TestPlatform.Extensions.HtmlLogger/HtmlLogger.cs

Lines 331 to 338 in ad32654

	htmlTransformer.Transform(XmlFilePath, HtmlFilePath);
	}
	catch (Exception ex)
	{
	EqtTrace.Error("HtmlLogger : Failed to populate html file. Exception : {0}",
	ex.ToString());
	ConsoleOutput.Instance.Error(false, string.Concat(HtmlResource.HtmlLoggerError), ex.Message);
	return;

Attaching a debugger I see the following stack trace:

 	System.Private.Xml.dll!System.Xml.XmlTextReaderImpl.Throw(System.Exception e) Line 2822	C#
 	System.Private.Xml.dll!System.Xml.XmlTextReaderImpl.Throw(string res, string[] args) Line 2796	C#
>	System.Private.Xml.dll!System.Xml.XmlTextReaderImpl.ParseNumericCharRefInline(int startPos, bool expand, System.Text.StringBuilder internalSubsetBuilder, out int charCount, out System.Xml.XmlTextReaderImpl.EntityType entityType) Line 7433	C#
 	System.Private.Xml.dll!System.Xml.XmlTextReaderImpl.ParseCharRefInline(int startPos, out int charCount, out System.Xml.XmlTextReaderImpl.EntityType entityType) Line 7307	C#
 	System.Private.Xml.dll!System.Xml.XmlTextReaderImpl.ParseText(out int startPos, out int endPos, ref int outOrChars) Line 5687	C#
 	System.Private.Xml.dll!System.Xml.XmlTextReaderImpl.ParseText() Line 5533	C#
 	System.Private.Xml.dll!System.Xml.XmlTextReaderImpl.ParseElementContent() Line 4351	C#
 	System.Private.Xml.dll!System.Xml.XPath.XPathDocument.LoadFromReader(System.Xml.XmlReader reader, System.Xml.XmlSpace space) Line 285	C#
 	System.Private.Xml.dll!System.Xml.XPath.XPathDocument.XPathDocument(System.Xml.XmlReader reader, System.Xml.XmlSpace space) Line 76	C#
 	System.Private.Xml.dll!System.Xml.Xsl.Runtime.XmlQueryContext.ConstructDocument(object dataSource, string uriRelative, System.Uri uriResolved) Line 191	C#
 	System.Private.Xml.dll!System.Xml.Xsl.Runtime.XmlQueryContext.XmlQueryContext(System.Xml.Xsl.Runtime.XmlQueryRuntime runtime, object defaultDataSource, System.Xml.XmlResolver dataSources, System.Xml.Xsl.XsltArgumentList argList, System.Xml.Xsl.Runtime.WhitespaceRuleLookup wsRules) Line 74	C#
 	System.Private.Xml.dll!System.Xml.Xsl.Runtime.XmlQueryRuntime.XmlQueryRuntime(System.Xml.Xsl.Runtime.XmlQueryStaticData data, object defaultDataSource, System.Xml.XmlResolver dataSources, System.Xml.Xsl.XsltArgumentList argList, System.Xml.Xsl.Runtime.XmlSequenceWriter seqWrt) Line 81	C#
 	System.Private.Xml.dll!System.Xml.Xsl.XmlILCommand.Execute(object defaultDocument, System.Xml.XmlResolver dataSources, System.Xml.Xsl.XsltArgumentList argumentList, System.Xml.Xsl.Runtime.XmlSequenceWriter results) Line 89	C#
 	System.Private.Xml.dll!System.Xml.Xsl.XmlILCommand.Execute(object defaultDocument, System.Xml.XmlResolver dataSources, System.Xml.Xsl.XsltArgumentList argumentList, System.Xml.XmlWriter writer) Line 64	C#
 	System.Private.Xml.dll!System.Xml.Xsl.XslCompiledTransform.Transform(string inputUri, string resultsFile) Line 391	C#
 	Microsoft.VisualStudio.TestPlatform.Extensions.Html.TestLogger.dll!Microsoft.VisualStudio.TestPlatform.Extensions.HtmlLogger.HtmlTransformer.Transform(string xmlFile, string htmlFile) Line 33	C#
 	Microsoft.VisualStudio.TestPlatform.Extensions.Html.TestLogger.dll!Microsoft.VisualStudio.TestPlatform.Extensions.HtmlLogger.HtmlLogger.PopulateHtmlFile() Line 331	C#
 	Microsoft.VisualStudio.TestPlatform.Extensions.Html.TestLogger.dll!Microsoft.VisualStudio.TestPlatform.Extensions.HtmlLogger.HtmlLogger.TestRunCompleteHandler(object sender, Microsoft.VisualStudio.TestPlatform.ObjectModel.Client.TestRunCompleteEventArgs e) Line 309	C#

I assume we need to be escaping some data into the xml file that is getting passed into the XslTransform.

AB#1582509

[Evangelink]

I can still reproduce the issue on latest TP.

[danmoseley]

it forms this invalid XML

<TestRunDetails xmlns="http://schemas.datacontract.org/2004/07/Microsoft.VisualStudio.TestPlatform.Extensions.HtmlLogger.ObjectModel" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
	<ResultCollectionList>
		<TestResultCollection>
			<FailedResultList/>
			<Id>641133380</Id>
			<ResultList>
				<TestResult>
					<DisplayName>MyTest.UnitTest1.AllMatches(pattern: &quot;(\\uFFFE\\uFFFF)+&quot;, options: None, input: &quot;=====\xfffe\xffff\xfffe\xffff\xfffe====&quot;, matches: [(5, 4, &#xFFFE;&#xFFFF;&#xFFFE;&#xFFFF;)])</DisplayName>
					<Duration>1ms</Duration>
					<ErrorMessage i:nil="true"/>
					<ErrorStackTrace i:nil="true"/>
					<FullyQualifiedName>MyTest.UnitTest1.AllMatches</FullyQualifiedName>
					<InnerTestResults i:nil="true"/>
					<ResultOutcome>Passed</ResultOutcome>
					<TestResultId>3b55767a-cbec-2d65-7b69-2ece9a0cff45</TestResultId>
				</TestResult>
			</ResultList>
			<Source>C:\repro\1\bin\Debug\net6.0\proj.dll</Source>
		</TestResultCollection>
	</ResultCollectionList>
	<RunLevelMessageErrorAndWarning i:nil="true" xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays"/>
	<RunLevelMessageInformational xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
		<a:string>[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.3+1b45f5407b (64-bit .NET 6.0.18)</a:string>
		<a:string>[xUnit.net 00:00:01.27]   Discovering: proj</a:string>
		<a:string>[xUnit.net 00:00:07.78]   Discovered:  proj</a:string>
		<a:string>[xUnit.net 00:00:07.78]   Starting:    proj</a:string>
		<a:string>[xUnit.net 00:00:07.88]   Finished:    proj</a:string>
	</RunLevelMessageInformational>
	<Summary>
		<FailedTests>0</FailedTests>
		<PassPercentage>100</PassPercentage>
		<PassedTests>1</PassedTests>
		<SkippedTests>0</SkippedTests>
		<TotalRunTime>2m 2s</TotalRunTime>
		<TotalTests>1</TotalTests>
	</Summary>
</TestRunDetails>

these are invalid character references: 
per
https://www.w3.org/TR/2008/REC-xml-20081126/#NT-Char

[2]	Char	::=	#x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]	/* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */

[danmoseley]

It is running the XSL transform over an XML file created by DCS. DCS is tolerant of these invalid characters -- it will serialize as eg  and read that in successfully. I didn't track down where it is being tolerant but it is.

In default configuration XmlTextWriter will happily serialize the same way. But in default configuration XmlReader will not read it, and that is what is throwing here. It is necessary to set XmlWriterSettings.CheckCharacters to false. The fix should be to provide XslCompiledTransform with an XmlReader (rather than a path) with that flag set.

[danmoseley]

@nohwnd I believe https://github.com/microsoft/vstest/compare/main...danmoseley:vstest:fix.invalid.char?expand=1 is the likely fix, but the mocking is troublesome and I don't have more time to spend on it. Is this a test you can finish (as you are familiar with how testing is done) so we can put up a PR?

edit: I do not think this is mockable. I put up a PR without a test, validated locally.