Skip to content

Execute Behat tests only for a single content graph adapter at a time #4179

@bwaidelich

Description

@bwaidelich

With the introduction of the Neos.ContentGraph.PostgreSQLAdapter the behat tests have been adjusted to run against both implementations.

This was done for performance reasons mainly. But it has some drawbacks:

  • Tests are far more complex because they have to keep state of multiple adapters in memory and compare them individually
  • Failure resolution is more difficult because one has to determine the failing implementation from the (sometimes misleading) error message
  • Currently the way to skip tests for certain adapters is to use the @adapters tag in the feature files – this concept can't be used for future 3rd party implementations
  • Tests for multiple adapters can't be executed in parallel

I would suggest to simplify the implementation again and allow to specify the adapter via behat.yml configuration.
To allow certain tests to be skipped during development, we could either use an allow (or deny) list of tests in the configuration or introduce feature specific annotations (like @features=contentSubgraph.findDescendantNodes, contentSubgraph.findChildNodes or similar).

Note: Previously our CI took ~1.25 hrs to execute behat tests (for PHP 8.1 and 8.2 each). With the sync mode (activated with #4110) they take less than 15 minutes. Besides we could probably execute them in parallel

Metadata

Metadata

Assignees

Type

No type

Projects

Status

Done ✅

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions