KafkaCompanion no longer works with DevServices (3.25.0)

[vonatzigenc]

Describe the bug

KafkaCompanion, which is initialized via @WithTestResource(value = KafkaCompanionResource.class), no longer works correctly since upgrading from Quarkus 3.24.5 to 3.25.0.

On the one hand, the “CommonClientConfig” is no longer completely filled, and on the other hand, additional TestContainers are started.

It appears that the property “kafka.bootstrap.servers” is not set correctly from the perspective of KafkaCompanionResource, and it therefore ignores the started DevServices.

Reproducer for the issue: https://github.com/vonatzigenc/reproducer_quarkus_kafka_companion

Expected behavior

KafkaCompanion is initialized correctly for DevServices.

Actual behavior

• KafkaCompanion is not initialized correctly for DevServices.
• In addition to DevServices, a “quay.io/strimzi-test-container/test-container:0.109.2-kafka-3.9.0” is started.

How to Reproduce?

Preproducer repo: https://github.com/vonatzigenc/reproducer_quarkus_kafka_companion#

• Reproducer is based on kafka-avro-schema-quickstart with added quarkus-test-kafka-companion and simplified test
• Execute MovieWithKafkaCompanionTest

(With downgrade to 3.24.5 the test is green

Output of uname -a or ver

No response

Output of java -version

openjdk version "21.0.7" 2025-04-15 LTS

Quarkus version or git rev

3.25.0

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

[quarkus-bot]

/cc @alesj (kafka), @cescoffier (kafka), @geoand (devservices), @ozangunalp (kafka)

[gsmet]

Also cc @holly-cummins .

[vonatzigenc]

It looks to me like a more general problem with DevServicesContext and the “applicationConfigProvider” in “DevServicesResultBuildItem”.

The configuration is read in DevServicesConfigBuildStep before the applicationConfigProvider was used.

For example, the bootstrap URL set in DevServicesKafkaProcessor is missing.

However, I cannot judge where/what needs to be corrected.

[PhorkysXT]

I can confirm the issue with additional testcontainers starting, as our application exhibits the same behavior:
Pulling docker image: quay.io/strimzi-test-container/test-container:0.109.2-kafka-3.9.0

sadly this update breaks our build 😞

[gsmet]

My wild guess is that DevServicesConfigBuildStep should use Map<String, String> getConfig(Startable startable) to get the full config but I have no idea how to get the Startable from there.

And it might not be the right fix either.

I think we really need @holly-cummins or @ozangunalp to figure it out.

[vonatzigenc]

I tried to understand the processes a little better.

I noticed the following:

• The DevServices are started in StartupActionImpl at the beginning of run (or runMainClass / runMainClassBlocking).
• However, in the QuarkusTestExtension the TestResourceManager is initialised before that.

I'm not sure if this order is intentional/correct.

The simplest solution seems to be to start the DevServices earlier in the QuarkusTestExtension. (However, I'm probably overlooking some dependencies.)

The following adjustment seems to solve the problem with the properties.
vonatzigenc@36acb9d But I cannot assess the side effects.

[holly-cummins]

I tried to understand the processes a little better.

I noticed the following:

* The DevServices are started in [StartupActionImpl](https://github.com/quarkusio/quarkus/blob/main/core/deployment/src/main/java/io/quarkus/runner/bootstrap/StartupActionImpl.java) at the beginning of [`run`](https://github.com/quarkusio/quarkus/blob/main/core/deployment/src/main/java/io/quarkus/runner/bootstrap/StartupActionImpl.java#L313) (or [`runMainClass`](https://github.com/quarkusio/quarkus/blob/main/core/deployment/src/main/java/io/quarkus/runner/bootstrap/StartupActionImpl.java#L112) / [`runMainClassBlocking`](https://github.com/quarkusio/quarkus/blob/main/core/deployment/src/main/java/io/quarkus/runner/bootstrap/StartupActionImpl.java#L221)).

* However, in the QuarkusTestExtension the [TestResourceManager is initialised](https://github.com/quarkusio/quarkus/blob/main/test-framework/junit5/src/main/java/io/quarkus/test/junit/QuarkusTestExtension.java#L210) before that.

I'm not sure if this order is intentional/correct.

The simplest solution seems to be to start the DevServices earlier in the QuarkusTestExtension. (However, I'm probably overlooking some dependencies.)

Thanks for the analysis and the reproducer! Dev services used to start in the augmentation phases, but they're now supposed to start in a post-augmentation pre-startup phase. Not all dev services have been converted to the new model, but Kafka has. Anything in the augmentation phase which uses the output of dev service startup needs to do it in a lazy way (by passing functions). Anything using dev service output in the startup phase should be unaffected.

I can see (thanks to your analysis) that the test resource manager is also being initialised in the same loosely-defined post-augmentation pre-startup phase. I'd need a bit more coffee (or possibly some experimentation) to decide if it works to just swap the order of the dev services and test resource initialisation. Alternatively, adjusting config priorities might help, or introducing some laziness/function-passing somewhere into the chain might help.

The following adjustment seems to solve the problem with the properties. vonatzigenc@36acb9d But I cannot assess the side effects.

I'll have a look, thanks! vonatzigenc@36acb9d#diff-4b7eeebfe4cab671c063056d94f0690445ec6bf31c92c208a3b7abcb454e8a9bR310 is sort-of-equivalent to adjusting config priorities, and probably a bit less clean. But I need to sit down and think properly about the side effects of the start-order-swap. We have some situations where test resources are intended to use the output of dev services, and some where test resources are intended be used instead of dev services. Both of those should work fine with dev services starting second, but there might be other cases where it's the opposite.

[holly-cummins]

One thing that's a bit surprising is that we do (as you'd expect) have test coverage of the KafkaCompanion, and those tests are still passing. So I think there's some extra condition in the reproducer.

[vonatzigenc]

I found two cases with KafkaCompanion that still work:

1. DevService is disabled (like in integration-tests/kafka)
2. Avro isn't used (Additional TestContainers are started, but “my tests” have not failed either.)

Additionally: If the DevService is already running the DevServiceKafkaProcessor will set the containerAddress url directly as config and not via configProvider.

However, I only saw this in the code and did not analyze it further.

[holly-cummins]

I think it's Avro, not the dev services being disabled. I tried turning off dev services in your reproducer, and turning on dev services in the integration tests, and the failing test continued failing, and the passing test continued passing. That really surprises me, so I might try again tomorrow in case I'm doing something dumb and checking wrong.

I also looked at OpenTelemetryTestCase, and it seems to work with dev services enabled, but again, there's no Avro there. But I'm puzzled about the mechanism for Avro messing it up, since if the config isn't there yet, the config isn't there yet.

[vonatzigenc]

Can confirm, that it's the "avro" part, that will lead to the failing test.

The following workaround allows the test to be performed successfully in the reproducer: Commented out BeforeEach

However, one Strimzi TestContainer too many is still being started and used. The DevService for Kafka is also started, but not used.
( I've added a log statement in me reproducer: MovieResource )

Observed startup order of the Docker containers:

1. ryuk
2. Avro (apicurio)
3. ryuk (again)
4. Strimzi (from KafkaCompanionResource)
5. redpanda