InaccessibleObjectException when serializing JsonLayout with JDK17+

[asanguinetti]

Description

When using the JsonLayout in a very simple configuration such as this:

<Console name="Console" target="SYSTEM_OUT">
  <JsonLayout/>
</Console>

There is an InaccessibleObjectException when trying to instantiate a ContextDataSerializer because its constructor is protected:

ERROR An exception occurred processing Appender Console java.lang.reflect.InaccessibleObjectException: Unable to make protected org.apache.logging.log4j.core.jackson.ContextDataSerializer() accessible: module org.apache.logging.log4j.core does not "opens org.apache.logging.log4j.core.jackson" to module com.fasterxml.jackson.databind

We are aware that JsonLayout has been marked as deprecated and there is another alternative called JsonTemplateLayout. That one works. However, our customers are still using JsonLayout in their configurations and they would like to be able to keep doing so.

Configuration

Version: Tested in 2.23.1 but probably happens on any 2.x

Operating system: Doesn't seem to matter, but tested with macOS 14.6

JDK: Tested with Temurin-17.0.11+9

Logs

2024-08-08T18:40:40.439988Z main ERROR An exception occurred processing Appender Console java.lang.reflect.InaccessibleObjectException: Unable to make protected org.apache.logging.log4j.core.jackson.ContextDataSerializer() accessible: module org.apache.logging.log4j.core does not "opens org.apache.logging.log4j.core.jackson" to module com.fasterxml.jackson.databind
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:188)
	at java.base/java.lang.reflect.Constructor.setAccessible(Constructor.java:181)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.util.ClassUtil.checkAndFixAccess(ClassUtil.java:1005)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.util.ClassUtil.findConstructor(ClassUtil.java:577)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.util.ClassUtil.createInstance(ClassUtil.java:559)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializerInstance(DefaultSerializerProvider.java:144)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ser.BasicSerializerFactory.findSerializerFromAnnotation(BasicSerializerFactory.java:547)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ser.BeanSerializerFactory._constructWriter(BeanSerializerFactory.java:860)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ser.BeanSerializerFactory.findBeanProperties(BeanSerializerFactory.java:634)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ser.BeanSerializerFactory.constructBeanOrAddOnSerializer(BeanSerializerFactory.java:402)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ser.BeanSerializerFactory.findBeanOrAddOnSerializer(BeanSerializerFactory.java:295)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ser.BeanSerializerFactory._createSerializer2(BeanSerializerFactory.java:240)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:174)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:1525)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1473)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.SerializerProvider.findValueSerializer(SerializerProvider.java:578)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.SerializerProvider.findTypedValueSerializer(SerializerProvider.java:856)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:330)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1574)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ObjectWriter._writeValueAndClose(ObjectWriter.java:1275)
	at com.fasterxml.jackson.databind@2.17.2/com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:1114)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.layout.AbstractJacksonLayout.toSerializable(AbstractJacksonLayout.java:379)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.layout.JsonLayout.toSerializable(JsonLayout.java:390)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.layout.AbstractJacksonLayout.toSerializable(AbstractJacksonLayout.java:327)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.layout.JsonLayout.toSerializable(JsonLayout.java:69)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.layout.AbstractJacksonLayout.toSerializable(AbstractJacksonLayout.java:50)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.layout.AbstractStringLayout.toByteArray(AbstractStringLayout.java:295)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.layout.AbstractLayout.encode(AbstractLayout.java:207)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.layout.AbstractLayout.encode(AbstractLayout.java:36)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:227)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:220)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:211)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:160)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:133)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:124)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:88)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:705)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:663)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:639)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:575)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:92)
	at org.apache.logging.log4j.core@2.23.1/org.apache.logging.log4j.core.Logger.log(Logger.java:169)
	at org.apache.logging.log4j@2.23.1/org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2906)
	at org.apache.logging.log4j@2.23.1/org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2859)
	at org.apache.logging.log4j@2.23.1/org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2841)
	at org.apache.logging.log4j@2.23.1/org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2620)
	at org.apache.logging.log4j@2.23.1/org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2567)
	at org.apache.logging.log4j@2.23.1/org.apache.logging.log4j.spi.AbstractLogger.warn(AbstractLogger.java:3469)
	at log4j.json.test@1.0-SNAPSHOT/org.mule.tests.Main.main(Main.java:12)

Reproduction

Attaching a minimalistic Maven project that reproduces the issue.
log4j-jsonLayout-test.zip

[ppkarwasz]

@asanguinetti,

Thanks for the bug report and the minimal reproducible example. The JPMS access problems should be fixed by #2815.

Remark: the JsonLayout class will be removed in 3.x. However JTL already has an embedde JsonLayout.json template that mimics the way JsonLayout works.

Since Log4j plugins are retrieved by plugin name and not class name, we could create a new plugin called "JsonLayout" that returns a properly configured JsonTemplateLayout. @vy, what do you think?

[vy]

The JPMS access problems should be fixed by #2815.

Great job @ppkarwasz!

Since Log4j plugins are retrieved by plugin name and not class name, we could create a new plugin called "JsonLayout" that returns a properly configured JsonTemplateLayout. @vy, what do you think?

This is not possible, since, quoting from the JSON Template Layout manual:

[JsonLayout.json] Models the exact structure generated by the deprecated JsonLayout with the exception of thrown field. JsonLayout serializes the Throwable as is via Jackson ObjectMapper, whereas JsonLayout.json event template employs the StackTraceElementLayout.json stack trace template for stack traces to generate a document-store-friendly flat structure.

In short, the thrown property of the produced JSONs will differ.

I had thought of doing something similar for GELF Layout, but in that case, there is the compression option we need to deal with.

[asanguinetti]

Thank you guys for the quick response!

And also thanks for the JsonLayout.json hint. We could make the suggestion to our customers, with the disclaimer about Throwables. But as far as I understand, using JTL will require them to also add an additional dependency: log4j-layout-template-json. Not sure if that is going to be possible under our current classloading configuration, but in any case that would be for us to figure out. If we can have the original JsonLayout working with JPMS that would be a great relief.

Looking forward for 2.24.