xmlReportRemainsSameAfterReadWrite fails due to non-deterministic XML element order

[hwu2024]

Summary

Modules: optaplanner-benchmark

Steps to Reproduce

mvn -pl optaplanner-benchmark \
  edu.illinois:nondex-maven-plugin:2.2.1:nondex \
  -Dtest=org.optaplanner.benchmark.impl.result.PlannerBenchmarkResultTest#xmlReportRemainsSameAfterReadWrite

The test fails with an assertion like:

org.opentest4j.AssertionFailedError:

expected:
  "<?xml version="1.0" encoding="UTF-8"?><plannerBenchmarkResult>
      <name>2020-07-13_125809</name>
      <aggregation>false</aggregation>
      ...
  </plannerBenchmarkResult>"
but was:
  "<?xml version="1.0" encoding="UTF-8"?><plannerBenchmarkResult>
      <javaVersion>Java 1.8.0_252 (Oracle Corporation)</javaVersion>
      <warmUpTimeMillisSpentLimit>30</warmUpTimeMillisSpentLimit>
      ...
  </plannerBenchmarkResult>"

The content is the same, but the order of child elements under <plannerBenchmarkResult>, <solverBenchmarkResult>, and <unifiedProblemBenchmarkResult> differs.

Root Cause

BenchmarkResultIO.write() serializes the PlannerBenchmarkResult object graph to XML.

Some parts of that graph are backed by unordered collections like HashMap or HashSet.

The test currently uses: assertThat(jaxbString.trim()).isXmlEqualTo(originalXml.trim()); which effectively enforces a deterministic XML serialization order, causing failures when sibling elements appear in a different (but semantically equivalent) order.

Fixes

Compare the two XML documents semantically while ignoring:

• insignificant whitespace

• the order of child nodes

I am currently working on the fixes and will later provide a PR related to this issue.