Jandex: reindex if the index is too new #47283

Ladicek · 2025-04-10T08:40:34Z

We've been bitten by this issue many times in the past: if a dependency upgrades to a newer Jandex than what Quarkus contains, build fails with an UnsupportedVersion exception.

This commit is a proper solution: if the exception is thrown, we swallow it and reindex the archive with the Jandex version that Quarkus incorporates.

Further, this commit increments the minimum Jandex version we "trust" from Jandex 2.1 to Jandex 3.0. We rely on new features from Jandex 2.4 and 3.0 on many places, so it makes sense to make 3.0 the new minimum. The changes added in 3.2 and 3.3 are also important, but are not used heavily yet.

Ladicek · 2025-04-10T08:41:44Z

core/deployment/src/main/java/io/quarkus/deployment/index/IndexingUtil.java


    private static class MetaInfJandexReader implements Function<PathVisit, Index> {
-        private static MetaInfJandexReader instance;
+        private final String treeDescription;


I added this because visit.getRoot() often returns just / (e.g. in case of a JAR from local Maven repo), so that's quite useless. PathTree.toString() is much more descriptive (e.g. in case of a JAR from local Maven repo, it returns the full path).

Ladicek · 2025-04-10T08:42:34Z

I think we would like to backport this to 3.20, but I'll leave the decision up to you :-)

gastaldi

LGTM, +1 to backport this to 3.20

gsmet · 2025-04-10T11:58:25Z

The tests are failing with:

Expecting empty but was: ["Reindexing %s:%s, at least Jandex 3.0 must be used to index an application dependency (index version is %s)",
    "Reindexing %s:%s, at least Jandex 3.0 must be used to index an application dependency (index version is %s)",
    "Reindexing %s:%s, at least Jandex 3.0 must be used to index an application dependency (index version is %s)",
    "Reindexing %s:%s, at least Jandex 3.0 must be used to index an application dependency (index version is %s)",
    "Reindexing %s:%s, at least Jandex 3.0 must be used to index an application dependency (index version is %s)"]

I think we have a couple of actions here:

wondering if the messages should include the parameters? - not sure if it's worth it, maybe it's just the toString() being conservative
we should probably exclude these messages for these tests
we need a list of the artifacts non indexed with Jandex 3+ and we need to fix them

Ladicek · 2025-04-10T12:27:40Z

wondering if the messages should include the parameters? - not sure if it's worth it, maybe it's just the toString() being conservative

The messages include the parameters, because interpolation happens further down the line.

we should probably exclude these messages for these tests

I've got a commit locally that does just that. Testing now if I've covered all tests.

we need a list of the artifacts non indexed with Jandex 3+ and we need to fix them

I think these PRs should fix this:

I'll check the log if there's more, just to be sure.

gsmet · 2025-04-10T12:32:08Z

I see 4 messages, two are coming from the same project?

Ladicek · 2025-04-10T12:33:32Z

These are the messages I see in one randomly selected test:

2025-04-10T10:15:04.2459612Z 2025-04-10 10:15:04,118 WARN  [io.qua.dep.index] (build-17) Reindexing /home/runner/.m2/repository/io/smallrye/common/smallrye-common-annotation/2.11.0/smallrye-common-annotation-2.11.0.jar:/META-INF/jandex.idx, at least Jandex 3.0 must be used to index an application dependency (index version is 10)
2025-04-10T10:15:04.2462815Z 2025-04-10 10:15:04,129 WARN  [io.qua.dep.index] (build-17) Reindexing /home/runner/.m2/repository/io/smallrye/smallrye-context-propagation-storage/2.2.0/smallrye-context-propagation-storage-2.2.0.jar:/META-INF/jandex.idx, at least Jandex 3.0 must be used to index an application dependency (index version is 10)
2025-04-10T10:15:04.2466251Z 2025-04-10 10:15:04,130 WARN  [io.qua.dep.index] (build-17) Reindexing /home/runner/.m2/repository/io/smallrye/reactive/smallrye-reactive-converter-api/3.0.1/smallrye-reactive-converter-api-3.0.1.jar:/META-INF/jandex.idx, at least Jandex 3.0 must be used to index an application dependency (index version is 10)
2025-04-10T10:15:04.2468939Z 2025-04-10 10:15:04,130 WARN  [io.qua.dep.index] (build-17) Reindexing /home/runner/.m2/repository/io/smallrye/reactive/smallrye-reactive-converter-mutiny/3.0.1/smallrye-reactive-converter-mutiny-3.0.1.jar:/META-INF/jandex.idx, at least Jandex 3.0 must be used to index an application dependency (index version is 10)
2025-04-10T10:15:04.2471537Z 2025-04-10 10:15:04,139 WARN  [io.qua.dep.index] (build-17) Reindexing /home/runner/.m2/repository/io/smallrye/common/smallrye-common-vertx-context/2.11.0/smallrye-common-vertx-context-2.11.0.jar:/META-INF/jandex.idx, at least Jandex 3.0 must be used to index an application dependency (index version is 10)

These are all from the 3 projects I linked above.

gastaldi · 2025-04-10T12:38:44Z

Perhaps we can remove the :/META-INF/jandex.idx from the logs, as we know they won't change

Ladicek · 2025-04-10T12:42:31Z

Good point @gastaldi, will do!

gsmet · 2025-04-10T12:56:17Z

core/deployment/src/main/java/io/quarkus/deployment/index/IndexingUtil.java

                        }
+                        return reader.read();
+                    } catch (UnsupportedVersion e) {
+                        log.warnf("Reindexing %s, the index is too new (%s)", file, e.getMessage());


I'm not that excited about the message. Maybe ..., the index format of %s is too new for the Jandex version used by your application. Please report it to the author of this jar..

Or similar. The problem is about the index format, not the index and we should really avoid cryptic messages. And also, we should be clear that it's something to fix at the library level, not in your app.

The last part might be something to include in the previous message too.

Feel free to improve the wording, it's just to give a general direction.

Obviously, if we adjust things here, we need to do the same below.

I'll try to improve this.

We've been bitten by this issue many times in the past: if a dependency upgrades to a newer Jandex than what Quarkus contains, build fails with an `UnsupportedVersion` exception. This commit is a proper solution: if the exception is thrown, we swallow it and reindex the archive with the Jandex version that Quarkus incorporates. Further, this commit increments the minimum Jandex version we "trust" from Jandex 2.1 to Jandex 3.0. We rely on new features from Jandex 2.4 and 3.0 on many places, so it makes sense to make 3.0 the new minimum. The changes added in 3.2 and 3.3 are also important, but are not used heavily yet.

gsmet · 2025-04-10T15:04:16Z

@Ladicek it would be nice if we could have releases for the jars mentioned above before we push 3.22 so that we avoid reindexing every time we build an application.

Ladicek · 2025-04-10T15:13:39Z

That should be doable.

quarkus-bot · 2025-04-10T17:13:43Z

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit 97572fa.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

You can consult the Develocity build scans.

Flaky tests - Develocity

⚙️ JVM Tests - JDK 21

📦 extensions/smallrye-reactive-messaging/deployment

✖ io.quarkus.smallrye.reactivemessaging.hotreload.ConnectorChangeTest.testUpdatingConnector - History

Expecting actual: ["-6","-7","-8","-10","-11","-12","-13","-14"] to start with: ["-6", "-7", "-8", "-9"] - java.lang.AssertionError

java.lang.AssertionError: 

Expecting actual:
  ["-6","-7","-8","-10","-11","-12","-13","-14"]
to start with:
  ["-6", "-7", "-8", "-9"]

	at io.quarkus.smallrye.reactivemessaging.hotreload.ConnectorChangeTest.testUpdatingConnector(ConnectorChangeTest.java:41)

⚙️ JVM Integration Tests - JDK 17

📦 integration-tests/opentelemetry-grpc-only

✖ io.quarkus.it.opentelemetry.grpc.HelloGrpcClientTest.testHello - History

java.lang.RuntimeException: Failed to start quarkus - java.lang.RuntimeException

java.lang.RuntimeException: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.test.junit.QuarkusTestExtension.throwBootFailureException(QuarkusTestExtension.java:682)
	at io.quarkus.test.junit.QuarkusTestExtension.interceptTestClassConstructor(QuarkusTestExtension.java:780)
	at java.base/java.util.Optional.orElseGet(Optional.java:364)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)

Ladicek requested review from gastaldi and gsmet April 10, 2025 08:40

quarkus-bot bot added the area/core label Apr 10, 2025

Ladicek commented Apr 10, 2025

View reviewed changes

gastaldi approved these changes Apr 10, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

gsmet reviewed Apr 10, 2025

View reviewed changes

Ladicek force-pushed the jandex-auto-reindexing branch from 649f8c4 to 97572fa Compare April 10, 2025 13:54

quarkus-bot bot added the area/hibernate-orm Hibernate ORM label Apr 10, 2025

gsmet approved these changes Apr 10, 2025

View reviewed changes

gsmet added the triage/waiting-for-ci Ready to merge when CI successfully finishes label Apr 10, 2025

quarkus-bot bot added the triage/flaky-test label Apr 10, 2025

gsmet merged commit 7517f7e into quarkusio:main Apr 10, 2025
56 checks passed

quarkus-bot bot added this to the 3.22 - main milestone Apr 10, 2025

quarkus-bot bot removed the triage/waiting-for-ci Ready to merge when CI successfully finishes label Apr 10, 2025

Ladicek deleted the jandex-auto-reindexing branch April 11, 2025 06:35

Ladicek mentioned this pull request Apr 14, 2025

Bump all SmallRye projects that have a new Jandex index #47342

Merged

gsmet added triage/backport triage/backport-3.20 labels Apr 14, 2025

gsmet modified the milestones: 3.22 - main, 3.21.3 Apr 15, 2025

gsmet removed the triage/backport label Apr 15, 2025

jmartisk modified the milestones: 3.21.3, 3.20.1 May 6, 2025

jmartisk removed the triage/backport-3.20 label May 6, 2025

Jandex: reindex if the index is too new #47283

Jandex: reindex if the index is too new #47283

Uh oh!

Conversation

Ladicek commented Apr 10, 2025

Uh oh!

Ladicek Apr 10, 2025

Choose a reason for hiding this comment

Uh oh!

Ladicek commented Apr 10, 2025

Uh oh!

gastaldi left a comment

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

gsmet commented Apr 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ladicek commented Apr 10, 2025

Uh oh!

gsmet commented Apr 10, 2025

Uh oh!

Ladicek commented Apr 10, 2025

Uh oh!

gastaldi commented Apr 10, 2025

Uh oh!

Ladicek commented Apr 10, 2025

Uh oh!

gsmet Apr 10, 2025

Choose a reason for hiding this comment

Uh oh!

Ladicek Apr 10, 2025

Choose a reason for hiding this comment

Uh oh!

Ladicek Apr 10, 2025

Choose a reason for hiding this comment

Uh oh!

gsmet commented Apr 10, 2025

Uh oh!

Ladicek commented Apr 10, 2025

Uh oh!

quarkus-bot bot commented Apr 10, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Status for workflow Quarkus CI

Flaky tests - Develocity

⚙️ JVM Tests - JDK 21

📦 extensions/smallrye-reactive-messaging/deployment

⚙️ JVM Integration Tests - JDK 17

📦 integration-tests/opentelemetry-grpc-only

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gsmet commented Apr 10, 2025 •

edited

Loading

quarkus-bot bot commented Apr 10, 2025 •

edited by github-actions bot

Loading

Status for workflow `Quarkus CI`