tappy fails to parse apparently valid TAP stream

[rossburton]

With the following TAP stream in a file:

Running test: json-glib-1.0/serialize-complex.test
# random seed: R02Sf8ed9453b07f85972a766377c0d776bb
1..1
# Start of serialize tests
ok 1 /serialize/gobject-boxed
# End of serialize tests
Running test: json-glib-1.0/reader.test
# random seed: R02S5b7da81711bc09111435d4be146a0e53
1..4
# Start of reader tests
ok 1 /reader/base-array
ok 2 /reader/base-object
ok 3 /reader/level
# Bug Reference: http://bugzilla.gnome.org/show_bug.cgi?id=758580
ok 4 /reader/null-value
# End of reader tests

ross@bob~ $ tappy --verbose <tap2
<file=stream>
/serialize/gobject-boxed ... ok
<file=stream>
/reader/base-array ... ok
<file=stream>
/reader/base-object ... ok
<file=stream>
/reader/level ... ok
<file=stream>
/reader/null-value ... ok
<file=stream>
Only one plan line is permitted per file. ... FAIL

======================================================================
FAIL: <file=stream>
Only one plan line is permitted per file.
----------------------------------------------------------------------

----------------------------------------------------------------------
Ran 6 tests in 0.000s

FAILED (failures=1)

[rossburton]

FWIW, node-tap parses the same data successfully.

ross@bob~ $ tap - <tap2
 PASS  TAP
 ✓ /serialize/gobject-boxed

 PASS  TAP
 ✓ /reader/base-array

 PASS  TAP
 ✓ /reader/base-object

 PASS  TAP
 ✓ /reader/level

 PASS  TAP
 ✓ /reader/null-value



  🌈 SUMMARY RESULTS 🌈


Suites:   0 of 0 completed
Asserts:  5 passed, of 5
Time:     82.385ms

[mblayman]

Technically, tappy is correct on this matter. The spec does not support multiple plan lines. I brought up this exact subject in 2015 and the proposal was shot down (TestAnything/testanything.github.io#36 (comment)) for some valid reasons.

Part of what tappy does is validate if the TAP data adheres to the spec. In the case of multiple plans, that is out of spec so tappy fails.

Issue #39 is for opting out of parsing errors. That could solve the failure, but I think it would lead to other errors because tappy would not be able to determine which is the correct plan count (since it would be the summation of the plans).

The tappy parser aims to be compliant with the spec. Since multiple plans are not permitted by the spec, I'm not sure if there is a correct course of action to fix this. I do not want to diverge from what the spec says.

It's been a long time since I tried, but I think this would work properly if each of those TAP outputs were in separate files and passed into tappy.

[rossburton]

Sigh. I'm really going off TAP. It's most like it was thrown together one night when drunk and retroactively justified.

[mblayman]

I hear you for sure. It definitely doesn't handle all the cases that users sometimes need. I'm going to close this one because there's not much I can do about it and stay compliant with the spec.