Invalid Sequence Coverage and Crap Score when merging reports.

[chrisj-unity]

Hi,

When merging two OpenCover reports together using the Report Generator, I've noticed that the Sequence Coverage and Crap Score calculations in the merged reports are incorrect.

For example, with this simple test class:

public class TestClass
{
    public static int DoSomething(bool value)
    {
        if (value)
            return 1;
        
        return 2;
    }
}

Calling DoSomething(true) produces the following OpenCover xml file:

<?xml version="1.0" encoding="utf-8"?>
<CoverageSession xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Version="0.0.0.0">
  <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="0" maxCrapScore="0" minCrapScore="0" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" />
  <Modules>
    <Module>
      <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="0" maxCrapScore="0" minCrapScore="0" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" />
      <ModuleTime>0001-01-01T00:00:00</ModuleTime>
      <ModuleName>Assets</ModuleName>
      <Files>
        <File uid="1" fullPath="/Users/chrisj/Chris/CrapScoreTest/TestClass.cs" />
      </Files>
      <Classes>
        <Class>
          <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="0" maxCrapScore="0" minCrapScore="0" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" />
          <FullName>TestClass</FullName>
          <Methods>
            <Method visited="true" cyclomaticComplexity="2" nPathComplexity="0" sequenceCoverage="80.0" branchCoverage="0" crapScore="2.03" isConstructor="false" isStatic="true" isGetter="false" isSetter="false">
              <Summary numSequencePoints="5" visitedSequencePoints="4" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="80.0" branchCoverage="0" maxCyclomaticComplexity="2" minCyclomaticComplexity="2" maxCrapScore="2.03" minCrapScore="2.03" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" />
              <MetadataToken>100663297</MetadataToken>
              <Name>DoSomething(System.Boolean)</Name>
              <FileRef uid="1" />
              <SequencePoints>
                <SequencePoint vc="1" uspid="1" ordinal="0" offset="0" sl="4" sc="5" el="4" ec="5" bec="0" bev="0" fileid="1" />
                <SequencePoint vc="1" uspid="2" ordinal="0" offset="1" sl="5" sc="9" el="5" ec="9" bec="0" bev="0" fileid="1" />
                <SequencePoint vc="1" uspid="3" ordinal="0" offset="6" sl="6" sc="13" el="6" ec="13" bec="0" bev="0" fileid="1" />
                <SequencePoint vc="0" uspid="4" ordinal="0" offset="10" sl="8" sc="9" el="8" ec="9" bec="0" bev="0" fileid="1" />
                <SequencePoint vc="1" uspid="5" ordinal="0" offset="14" sl="9" sc="5" el="9" ec="5" bec="0" bev="0" fileid="1" />
              </SequencePoints>
              <BranchPoints />
            </Method>
          </Methods>
        </Class>
      </Classes>
    </Module>
  </Modules>
</CoverageSession>

and when calling DoSomething(false) I'm getting this:

<?xml version="1.0" encoding="utf-8"?>
<CoverageSession xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Version="0.0.0.0">
  <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="0" maxCrapScore="0" minCrapScore="0" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" />
  <Modules>
    <Module>
      <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="0" maxCrapScore="0" minCrapScore="0" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" />
      <ModuleTime>0001-01-01T00:00:00</ModuleTime>
      <ModuleName>Assets</ModuleName>
      <Files>
        <File uid="1" fullPath="/Users/chrisj/Chris/CrapScoreTest/TestClass.cs" />
      </Files>
      <Classes>
        <Class>
          <Summary numSequencePoints="0" visitedSequencePoints="0" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="0" branchCoverage="0" maxCyclomaticComplexity="0" minCyclomaticComplexity="0" maxCrapScore="0" minCrapScore="0" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" />
          <FullName>TestClass</FullName>
          <Methods>
            <Method visited="true" cyclomaticComplexity="2" nPathComplexity="0" sequenceCoverage="80.0" branchCoverage="0" crapScore="2.03" isConstructor="false" isStatic="true" isGetter="false" isSetter="false">
              <Summary numSequencePoints="5" visitedSequencePoints="4" numBranchPoints="0" visitedBranchPoints="0" sequenceCoverage="80.0" branchCoverage="0" maxCyclomaticComplexity="2" minCyclomaticComplexity="2" maxCrapScore="2.03" minCrapScore="2.03" visitedClasses="0" numClasses="0" visitedMethods="0" numMethods="0" />
              <MetadataToken>100663297</MetadataToken>
              <Name>DoSomething(System.Boolean)</Name>
              <FileRef uid="1" />
              <SequencePoints>
                <SequencePoint vc="1" uspid="1" ordinal="0" offset="0" sl="4" sc="5" el="4" ec="5" bec="0" bev="0" fileid="1" />
                <SequencePoint vc="1" uspid="2" ordinal="0" offset="1" sl="5" sc="9" el="5" ec="9" bec="0" bev="0" fileid="1" />
                <SequencePoint vc="0" uspid="3" ordinal="0" offset="6" sl="6" sc="13" el="6" ec="13" bec="0" bev="0" fileid="1" />
                <SequencePoint vc="1" uspid="4" ordinal="0" offset="10" sl="8" sc="9" el="8" ec="9" bec="0" bev="0" fileid="1" />
                <SequencePoint vc="1" uspid="5" ordinal="0" offset="14" sl="9" sc="5" el="9" ec="5" bec="0" bev="0" fileid="1" />
              </SequencePoints>
              <BranchPoints />
            </Method>
          </Methods>
        </Class>
      </Classes>
    </Module>
  </Modules>
</CoverageSession>

If I use the report generator to merge the two files I see the following output.

The sequence points are merged correctly (i.e. all lines are green and have the correct visit counts), but in the metrics box, the Sequence coverage is only 80% and the Crap Score is 2.03, instead of the expected 100% and 2 respectively.

Thanks,
Chris.

[danielpalme]

Both XML coverage files contain the following element:

<Method visited="true" cyclomaticComplexity="2" nPathComplexity="0" sequenceCoverage="80.0" branchCoverage="0" crapScore="2.03" isConstructor="false" isStatic="true" isGetter="false" isSetter="false">

ReportGenerator relies on those values (sequenceCoverage="80.0" and crapScore="2.03") and renderes them in the report.
ReportGenerator does not recalculate those values when files are merged. It just uses the maximum value.
I'm aware that this is not 100% correct. But otherwise I would have to implement a (probably non trivial) merging algorithm for many metrics and coverage formats.

Perhaps it's better to generate just a single coverage file or to use the -mergeoutput option of OpenCover:
https://github.com/OpenCover/opencover/wiki/Usage

[Bolt54]

Wouldn't it make more sense to use the minimum CRAP score (i.e. the best score)? This would seem to match what is being done by taking the max code coverage value. I think this would make the report look more correct as the report with the higher coverage will have the lower CRAP score.

[danielpalme]

@Bolt54
You are right, that makes sense. I changed the behavior

New release contains the changes:
https://www.nuget.org/packages/ReportGenerator/4.1.9