Upgrading from 3.18.2 to 3.18.3 Results in OutOfMemoryError when using @QuarkusTest with Quarkus Junit 5

[brianwyka]

Describe the bug

When upgrading from Quarkus 3.18.2 to 3.18.3 our @QuarkusTests start to throw OutOfMemoryError.

It seems the issue is isolated in quarkus-junit5 dependency or one of its transitives, when I upgrade the rest of dependencies and leave that, things work fine.

Perhaps something to do with this change? But I'm not sure...
8203603

Expected behavior

Tests pass without OutOfMemoryError

Actual behavior

Tests annotated with @QuarkusTest hang and eventually run out of memory with different errors each time.
Example errors:

java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.arc.deployment.ArcProcessor#registerBeans threw an exception: java.lang.OutOfMemoryError: Java heap space

java.lang.RuntimeException: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.arc.deployment.ArcProcessor#registerBeans threw an exception: java.lang.OutOfMemoryError: Java heap space

	at io.quarkus.test.junit.QuarkusTestExtension.throwBootFailureException(QuarkusTestExtension.java:611)
	at io.quarkus.test.junit.QuarkusTestExtension.interceptTestClassConstructor(QuarkusTestExtension.java:695)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.arc.deployment.ArcProcessor#registerBeans threw an exception: java.lang.OutOfMemoryError: Java heap space

	at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:354)
	at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:271)
	at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:61)
	at io.quarkus.test.junit.QuarkusTestExtension.doJavaStart(QuarkusTestExtension.java:195)
	at io.quarkus.test.junit.QuarkusTestExtension.ensureStarted(QuarkusTestExtension.java:578)
	at io.quarkus.test.junit.QuarkusTestExtension.beforeAll(QuarkusTestExtension.java:628)
	... 1 more
Caused by: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.arc.deployment.ArcProcessor#registerBeans threw an exception: java.lang.OutOfMemoryError: Java heap space

	at io.quarkus.builder.Execution.run(Execution.java:124)
	at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79)
	at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:161)
	at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:350)
	... 6 more
Caused by: java.lang.OutOfMemoryError: Java heap space

How to Reproduce?

No response

Output of uname -a or ver

OSX and Linux

Output of java -version

17.0

Quarkus version or git rev

3.18.3

Build tool (ie. output of mvnw --version or gradlew --version)

Gradle 8.12.1

Additional information

I've also tried version 3.19.0 and the issue has NOT been resolved.

[geoand]

Thanks a lot for reporting!

We absolutely want to fix this, but we will almost certainly need to have a way to reproduce the issue.
It would also be very useful to know if the problem persists with 3.18.4 and 3.19.0 (for the time being, you'll have to use io.quarkus as the groupId to test this)

[brianwyka]

@geoand, also produces error on 3.18.4 and 3.19.0. I updated the description to indicate that the issue seems to be scoped to quarkus-junit5 updates.

[brianwyka]

I will try to come up with a reproducer.

[geoand]

🙏🏽

[gsmet]

What could help understanding what’s going on is to get a heap dump on OOM (I’m on my phone but you should find it easily) and have a closer look at what’s going on.

If your tests don’t contain any private information, we can give you our emails so that you can share the dump if you’re not used to the exercise.

Might give us some hints on how to reproduce it.

[brianwyka]

I did a quick test, and in our case, when I created a custom quarkus.test.class-orderer with the changes reverted from 8203603, the OutOfMemoryError goes away.

[geoand]

Thanks for that information!

That confirms that we are keeping something around that we shouldn't be. If you provide a heap dump or a sample project, I am pretty sure we can pinpoint the problem and fix it

[gsmet]

Maybe I should revert the change in the meantime? I still have time before I push 3.19.1 tomorrow.

[gsmet]

That being said, we probably want to get to the bottom of it so if we get more reports, it might help.