Skip to content

Conversation

@darvld
Copy link
Member

@darvld darvld commented Sep 8, 2025

Ready for review Powered by Pull Request Badge

Summary

This PR reworks our current test runner architecture, decoupling it from polyglot code to allow more flexible test features, such as host-side JVM testing (as opposed to just Espresso-run).

The main principle behind the new implementation is that tests are no longer executable on their own, a TestCase is now used to represent information that can be used by a TestDriver to execute the case instead. This allows JVM tests to be represented as data (e.g. a qualified class name and method name), and then executed in different ways by specific drivers.

Some components from the previous version (like the TestingRegistrar) are kept to ease the transition and will be refactored in the near future to better fit their purpose.

Future work

A few changes were left out of this PR's scope to avoid feature creep:

  • Host-side JVM test driver
  • JUnit Jupiter test discovery/execution
  • Improved output for the elide test command; proper support for parallel tests output

Note: this PR does not fulfill #1580, but it addresses it in part. JUnit support and the tooling events API (coming soon) will fully close the issue.

@darvld darvld self-assigned this Sep 8, 2025
@darvld darvld added the tools Issues and PRs related to tooling label Sep 8, 2025
@darvld darvld requested a review from sgammon as a code owner September 8, 2025 17:21
@darvld darvld added tools:builder Stuff relating to Elide's builder tools:test-runner Stuff relating to Elide's test runner module:tooling Changes and issues relating to the Elide `tooling` module labels Sep 8, 2025
@darvld darvld force-pushed the feat/test-runner-v2 branch from 0bc576f to 28f3edf Compare September 8, 2025 17:33
@codecov
Copy link

codecov bot commented Sep 8, 2025

Codecov Report

❌ Patch coverage is 3.23276% with 449 lines in your changes missing coverage. Please review.
✅ Project coverage is 40.02%. Comparing base (be553b1) to head (cbf2060).
⚠️ Report is 8 commits behind head on main.

Files with missing lines Patch % Lines
...kotlin/elide/tool/cli/cmd/repl/ToolShellCommand.kt 8.19% 163 Missing and 5 partials ⚠️
...ain/kotlin/elide/tooling/runner/ElideTestRunner.kt 0.00% 70 Missing ⚠️
...in/kotlin/elide/tooling/jvm/JvmTestConfigurator.kt 0.00% 61 Missing ⚠️
...otlin/elide/tooling/testing/jvm/JUnitTestDriver.kt 0.00% 31 Missing ⚠️
...otlin/elide/tooling/testing/MutableTestRegistry.kt 0.00% 19 Missing ⚠️
...in/kotlin/elide/tooling/config/TestConfigurator.kt 0.00% 17 Missing ⚠️
...kotlin/elide/tooling/testing/jvm/JUnitTestSuite.kt 0.00% 12 Missing ⚠️
...in/elide/tooling/testing/jvm/GuestJvmTestDriver.kt 0.00% 11 Missing ⚠️
...rc/main/kotlin/elide/tooling/testing/TestDriver.kt 0.00% 8 Missing ⚠️
...kotlin/elide/tooling/testing/TestDriverRegistry.kt 0.00% 7 Missing ⚠️
... and 11 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1625      +/-   ##
==========================================
+ Coverage   39.97%   40.02%   +0.04%     
==========================================
  Files         784      790       +6     
  Lines       37459    37414      -45     
  Branches     5300     5313      +13     
==========================================
  Hits        14976    14976              
+ Misses      20691    20647      -44     
+ Partials     1792     1791       -1     
Flag Coverage Δ
jvm 40.02% <3.23%> (+0.04%) ⬆️
lib 40.02% <3.23%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...rc/main/kotlin/elide/tooling/builder/TestDriver.kt 0.00% <0.00%> (ø)
...main/kotlin/elide/tooling/testing/TestRunResult.kt 0.00% <0.00%> (ø)
...ide/runtime/intrinsics/testing/TestingRegistrar.kt 30.35% <0.00%> (ø)
...n/kotlin/elide/tooling/config/TestConfigurators.kt 0.00% <0.00%> (ø)
.../main/kotlin/elide/tooling/testing/TestRegistry.kt 0.00% <0.00%> (ø)
.../src/main/kotlin/elide/tooling/testing/TestNode.kt 0.00% <0.00%> (ø)
...n/kotlin/elide/tooling/config/TestConfiguration.kt 0.00% <0.00%> (ø)
...in/kotlin/elide/tooling/testing/DynamicTestCase.kt 0.00% <0.00%> (ø)
...c/main/kotlin/elide/tooling/testing/TestOutcome.kt 0.00% <0.00%> (ø)
...kotlin/elide/tooling/testing/TestDriverRegistry.kt 0.00% <0.00%> (ø)
... and 11 more

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update be553b1...cbf2060. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@darvld darvld force-pushed the feat/test-runner-v2 branch 4 times, most recently from c54c024 to 1e8a542 Compare September 16, 2025 19:44
@darvld darvld force-pushed the feat/test-runner-v2 branch from 1e8a542 to cbf2060 Compare September 18, 2025 16:16
@darvld darvld enabled auto-merge (squash) September 18, 2025 16:18
@darvld darvld merged commit 23a67e7 into main Sep 18, 2025
17 checks passed
@sgammon sgammon mentioned this pull request Sep 21, 2025
31 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:tooling Changes and issues relating to the Elide `tooling` module tools:builder Stuff relating to Elide's builder tools:test-runner Stuff relating to Elide's test runner tools Issues and PRs related to tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants