Jetty12 (Jakarta EE8) upgrade #18424

capistrant · 2025-08-21T03:19:45Z

Description

Upgrade to Jetty 12

Upgrades our jetty dependency to jetty 12.0.x (12.0.25 at time of writing). Uses Jakarta EE8 compatible jetty 12 to enable a less intrusive upgrade.

Avatica Changes

Calcite Avatica has a hard dependency on jetty9 still and there is no clear timeline on their upgrade plans. To avoid this being a blocker we have reduced our dependency footprint on Avatica in this PR by bringing the Jetty handler implementations for Druid JDBC while retaining compatibility with the Avatica JDBC client.

Hadoop Compatibility

Hadoop ecosystem also comes with lots of dependencies on jetty9 and java11 support. This section lays out implications for hadoop support

Deep storage

Confirmed functionality using Docker IT env

index_hadoop

Confirmed functionality using Docker IT env. We will want more expansive testing here. I tried to slim down the code that sets up the injector, but since the task needs to load all of the extensions, I ended up falling back to an ugly duplication of the service injector creation, just without jetty. I'm sure it can be improved if we want to.

Java 11 Support Implications

Jetty12 deps are in java 17 class file version, so running druid services on java11 seems out of the question. If this is the case, java 11 support will have to be dropped in 35.

Future Work

Explore jetty 12 features and integrate them where appropriate

Release note

Upgrading Druid to Jetty 12 requires dropping Java 11 runtime support for Druid. The one exception to this is that Hadoop Ingestion can continue to run on Java 11 YARN clusters because the Jetty 12 dependency can be excluded from that runtime classpath safely.

A new server configuration option has been added with this PR: druid.server.http.uriCompliance. Jetty 12 by default has strict enforcement of RFC3986 URI format. This is a change from Jetty 9. To retain compatibility with legacy Druid, we default this config to LEGACY, which uses the more permissive URI format enforcement that jetty9 used. If the cluster you operate does not require LEGACY compatibility, it is recommended to use the upstream jetty default of RFC3986 in your Druid deployment. Please see jetty documentation for more info.

Upgrade Note

Java 11 runtime no longer supported. Java 17 is the only officially supported and recommended runtime environment for Druid 35

Key changed/added classes in this PR

PR is mostly a lot of tiny changes and tweaks. But some places to get close look could be:

JDBC related files
- sql/src/main/java/org/apache/druid/sql/avatica/DruidAvaticaJsonHandler.java
- sql/src/main/java/org/apache/druid/sql/avatica/DruidAvaticaProtobufHandler.java
Hadoop Indexing related files
- indexing-hadoop/src/main/java/org/apache/druid/indexer/HadoopDruidIndexerConfig.java
- indexing-hadoop/src/main/java/org/apache/druid/indexer/JobHelper.java
- indexing-hadoop/src/main/java/org/apache/druid/indexer/Utils.java
- server/src/main/java/org/apache/druid/initialization/CoreInjectorBuilder.java
- server/src/main/java/org/apache/druid/initialization/Initialization.java
- server/src/main/java/org/apache/druid/initialization/ServerInjectorBuilder.java
server/src/main/java/org/apache/druid/guice/http/JettyHttpClientModule.java
server/src/main/java/org/apache/druid/server/initialization/jetty/JettyServerInitUtils.java
server/src/main/java/org/apache/druid/server/initialization/jetty/JettyServerModule.java

This PR has:

…xed up

…ully working

server/src/main/java/org/apache/druid/initialization/ServerInjectorBuilder.java

+    // Create the core set of modules shared by all services.
+    // Here and below, the modules are filtered by the load modules list and
+    // the set of roles which this server provides.
+    CoreInjectorBuilder coreBuilder = new CoreInjectorBuilder(childInjector, nodeRoles).forServerWithoutJetty();


…llow backwards compatibility

Copilot

Pull Request Overview

This PR upgrades Apache Druid from Jetty 9 to Jetty 12 with Jakarta EE8 compatibility to minimize breaking changes while modernizing the web server infrastructure. Since Jetty 12 requires Java 17 runtime, Java 11 support is being dropped except for Hadoop ingestion tasks which can continue running on Java 11 YARN clusters.

Key changes:

Updates Jetty dependencies from 9.4.x to 12.0.25 with Jakarta EE8 modules
Replaces Avatica dependencies with custom JDBC handlers to avoid compatibility issues
Adds configuration for URI compliance to maintain backward compatibility with legacy URI formats

Reviewed Changes

Copilot reviewed 73 out of 73 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
pom.xml	Updates Jetty version to 12.0.25, switches to EE8 modules, adds servlet-api exclusions
licenses.yaml	Updates license information for new Jetty and Avatica versions
server/src/main/java/org/apache/druid/server/initialization/ServerConfig.java	Adds uriCompliance configuration property with LEGACY default
server/src/main/java/org/apache/druid/server/initialization/jetty/JettyServerModule.java	Updates Jetty server configuration for v12 API changes
sql/src/main/java/org/apache/druid/sql/avatica/*.java	Replaces Avatica handler inheritance with custom implementations
services/src/main/java/org/apache/druid/server/AsyncQueryForwardingServlet.java	Updates for new Jetty client API
indexing-hadoop/src/main/java/org/apache/druid/indexer/*.java	Removes Jetty dependencies and uses standard HTTP client for Hadoop compatibility

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-08-27T06:22:35Z

...in/java/org/apache/druid/server/initialization/jetty/StandardResponseHeaderFilterHolder.java

+      if (serverResponse.getHeaders().contains(headerName) && proxyResponse.containsHeader(headerName)) {
+        // In Jetty 12 EE8, use the standard servlet API method to remove headers
+        proxyResponse.setHeader(headerName, null);


Setting a header to null may not reliably remove it across all servlet implementations. Consider using a more explicit removal method or documenting this behavior's dependency on the specific servlet container.

Suggested change

if (serverResponse.getHeaders().contains(headerName) && proxyResponse.containsHeader(headerName)) {

// In Jetty 12 EE8, use the standard servlet API method to remove headers

proxyResponse.setHeader(headerName, null);

proxyResponse.removeHeader(headerName);

I guess I need to dig deeper into this. I don't think we can do the cast to a jetty response object like we used to. And I don't want to reach into the ee8 internals either, even if something in there would provide an api for removing a header.

Copilot · 2025-08-27T06:22:35Z

indexing-hadoop/src/main/java/org/apache/druid/indexer/JobHelper.java

+      // Keep Druid jetty jars out of the classpath. They are not runnable in the < 17 hadoop java runtime
+      if (jarFile.getName().endsWith(".jar") && !jarFile.getName().contains("jetty")) {


The string-based filtering !jarFile.getName().contains(\"jetty\") is fragile and could accidentally exclude non-Jetty JARs with 'jetty' in their name. Consider using a more specific pattern match or maintaining an explicit list of JARs to exclude.

I agree that this is suboptimal. I was in the "lets see if we can make it work" mindset. Maybe there is a more clean approach to filtering the jetty jars out.

Copilot · 2025-08-27T06:22:35Z

sql/src/main/java/org/apache/druid/sql/avatica/DruidAvaticaJsonHandler.java

+        }
+
+        callback.succeeded();
+        return true;


The callback.succeeded() is called unconditionally, even for non-POST requests that don't write any response. This should only be called after successfully writing a response, or the logic should be restructured to handle different response scenarios correctly.

Suggested change

}

callback.succeeded();

return true;

callback.succeeded();

return true;

} else {

// For non-POST requests, return 405 Method Not Allowed

response.setStatus(405);

response.getHeaders().put("Content-Type", "application/json;charset=utf-8");

String errorJson = "{\"error\": \"Method Not Allowed\"}";

response.write(true, ByteBuffer.wrap(errorJson.getBytes(StandardCharsets.UTF_8)), callback);

callback.succeeded();

return true;

}

latest commits should try to address this along with general lack of consistency in request handling between json and protobuf handlers.

kfaraz · 2025-09-03T03:22:39Z

Thanks for the PR, @capistrant ! I plan to finish a review today.

capistrant added 10 commits May 21, 2025 12:01

jetty 10 upgrade

ffdf8ee

checkpoint

f4eef97

compiling but broken cuz of at least one dep calcite depending on jetty9

63f23c9

Fix breakage in router

87e216d

Merge branch 'master' into jetty-update-august

c675117

Compilation fixes after merging master in. plus getting checkstyle fi…

acaf869

…xed up

Remove some of the avatica deps blocking jetty12

a741929

bump jetty to latest and fixup licenses.yaml

2536394

Merge branch 'master' into jetty12-ee8-upgrade

067b84b

fixup after merging master

a7a0134

capistrant added the WIP label Aug 21, 2025

github-actions bot added Area - Batch Ingestion Area - Querying Area - Dependencies Area - Ingestion labels Aug 21, 2025

capistrant added 3 commits August 20, 2025 23:06

Fixup some licenses reporting issues

aba3dd0

remove jetty dependency from indexing hadoop in attempt to keep it alive

8042ece

stop doing static checks and build with jdk11

11d2964

github-actions bot added the GHA label Aug 22, 2025

capistrant added 9 commits August 22, 2025 10:33

Modifications to index_hadoop to try and keep it alive

5f9ee32

Modify duplicate header removal strategy

1f13540

Remove dependency on non public jetty nested packages

74f5c09

Revert back to verbose and dirty Injector setup to get hadoop index f…

97e7649

…ully working

Fix http trailers for QueryResultPusher

0e08e10

Fix trailer related tests in QueryResourceTest

d2a5903

add some UTs

8ddb910

prevent banned dep from getting pulled in

a2a224d

Run rewrite rules mvn goal to fixup test files

8d9eb66

github-advanced-security bot found potential problems Aug 23, 2025

View reviewed changes

stop accounting for buggy mock impl in sql test

3660520

capistrant added 2 commits August 25, 2025 09:18

Add new http server config for Jetty UriCompliance mode. Default to a…

13e4f5c

…llow backwards compatibility

fix import order

6e8e5f5

github-actions bot added the Area - Documentation label Aug 25, 2025

fix spell checker flag

a66e475

capistrant changed the title ~~[WIP] Jetty12 (Jakarta EE8) upgrade~~ Jetty12 (Jakarta EE8) upgrade Aug 25, 2025

capistrant removed the WIP label Aug 25, 2025

cryptoe requested a review from Copilot August 26, 2025 05:08

This comment was marked as outdated.

Sign in to view

Fixup java docs links to use java17

9fb989d

abhishekagarwal87 requested a review from Copilot August 27, 2025 06:21

Copilot AI reviewed Aug 27, 2025

View reviewed changes

capistrant added 3 commits September 2, 2025 16:44

Unify approach to jetty handler for avatica json and protobuf

bdf6790

use 405 response code for avatica jdbc non post requests

99b0ebc

fix checkstyle

7aa43aa

capistrant added Release Notes Upgrade note Behavior change that requires an upgrade note labels Sep 2, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Jetty12 (Jakarta EE8) upgrade #18424

Jetty12 (Jakarta EE8) upgrade #18424

Uh oh!

capistrant commented Aug 21, 2025 •

edited

Loading

Uh oh!

Check notice

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Aug 27, 2025

Uh oh!

capistrant Sep 2, 2025

Uh oh!

Copilot AI Aug 27, 2025

Uh oh!

capistrant Sep 2, 2025

Uh oh!

Copilot AI Aug 27, 2025

Uh oh!

capistrant Sep 2, 2025

Uh oh!

kfaraz commented Sep 3, 2025

Uh oh!

Uh oh!

		// Keep Druid jetty jars out of the classpath. They are not runnable in the < 17 hadoop java runtime
		if (jarFile.getName().endsWith(".jar") && !jarFile.getName().contains("jetty")) {

Jetty12 (Jakarta EE8) upgrade #18424

Are you sure you want to change the base?

Jetty12 (Jakarta EE8) upgrade #18424

Uh oh!

Conversation

capistrant commented Aug 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Upgrade to Jetty 12

Avatica Changes

Hadoop Compatibility

Deep storage

index_hadoop

Java 11 Support Implications

Future Work

Release note

Upgrade Note

Key changed/added classes in this PR

Uh oh!

Check notice

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Aug 27, 2025

Choose a reason for hiding this comment

Uh oh!

capistrant Sep 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 27, 2025

Choose a reason for hiding this comment

Uh oh!

capistrant Sep 2, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 27, 2025

Choose a reason for hiding this comment

Uh oh!

capistrant Sep 2, 2025

Choose a reason for hiding this comment

Uh oh!

kfaraz commented Sep 3, 2025

Uh oh!

Uh oh!

capistrant commented Aug 21, 2025 •

edited

Loading