Native image - ClassNotFoundException: org.conscrypt.BufferAllocator

[rquinio]

Describe the bug

When upgrading from Quarkus 0.19.0 to 0.23.1 (GraalVM 19.0.2 to 19.2.0.1) I'm getting this error at native-image compilation:

[2019-09-26T17:55:00.105Z] [INFO] [io.quarkus.creator.phase.nativeimage.NativeImagePhase] Running Quarkus native-image plugin on OpenJDK 64-Bit GraalVM CE 19.2.0.1
[2019-09-26T17:55:00.105Z] [INFO] [io.quarkus.creator.phase.nativeimage.NativeImagePhase] /opt/graalvm/bin/native-image -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dio.netty.noUnsafe=true -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dio.netty.leakDetection.level=DISABLED -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true -J-Dvertx.disableDnsResolver=true --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar ojf-quarkus-packaging-runner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportUnsupportedElementsAtRuntime -H:+ReportExceptionStackTraces -H:+PrintAnalysisCallTree -H:-AddAllCharsets -H:EnableURLProtocols=http -H:+JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace
[2019-09-26T17:55:26.895Z] [quarkus-packaging-runner:1115]    classlist:  23,885.64 ms
[2019-09-26T17:55:26.895Z] [quarkus-packaging-runner:1115]        setup:     520.25 ms
[2019-09-26T17:55:26.895Z] Fatal error: java.lang.NoClassDefFoundError
[2019-09-26T17:55:26.895Z] 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[2019-09-26T17:55:26.895Z] 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[2019-09-26T17:55:26.895Z] 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[2019-09-26T17:55:26.895Z] 	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
[2019-09-26T17:55:26.896Z] 	at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
[2019-09-26T17:55:26.896Z] 	at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:461)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:310)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:448)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:113)
[2019-09-26T17:55:26.896Z] Caused by: java.lang.NoClassDefFoundError: org/conscrypt/BufferAllocator
[2019-09-26T17:55:26.896Z] 	at java.lang.Class.getDeclaredMethods0(Native Method)
[2019-09-26T17:55:26.896Z] 	at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
[2019-09-26T17:55:26.896Z] 	at java.lang.Class.getDeclaredMethods(Class.java:1975)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleDeletedClass(AnnotationSubstitutionProcessor.java:437)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.handleClass(AnnotationSubstitutionProcessor.java:270)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.substitute.AnnotationSubstitutionProcessor.init(AnnotationSubstitutionProcessor.java:230)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.NativeImageGenerator.createDeclarativeSubstitutionProcessor(NativeImageGenerator.java:875)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:824)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:524)
[2019-09-26T17:55:26.896Z] 	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:444)
[2019-09-26T17:55:26.896Z] 	at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
[2019-09-26T17:55:26.896Z] 	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
[2019-09-26T17:55:26.896Z] 	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
[2019-09-26T17:55:26.896Z] 	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
[2019-09-26T17:55:26.896Z] 	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
[2019-09-26T17:55:26.896Z] Caused by: java.lang.ClassNotFoundException: org.conscrypt.BufferAllocator
[2019-09-26T17:55:26.896Z] 	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
[2019-09-26T17:55:26.896Z] 	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
[2019-09-26T17:55:26.896Z] 	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
[2019-09-26T17:55:26.896Z] 	... 15 more
[2019-09-26T17:55:26.896Z] Error: Image build request failed with exit status 1

I tried to zgrep for conscrypt or org.conscrypt.BufferAllocator in target/lib, but didn't find any references in any jar.

Expected behavior

Should compile (was fine on Quarkus 0.19.0 + GraalVM 19.0.2).

To Reproduce

I don't have a reproducer I can share, but:

• I'm using the docker image quay.io/quarkus/centos-quarkus-maven:19.2.0.1 to build.
• My application uses the following extensions: quarkus-undertow, quarkus-spring-di, quarkus-resteasy, and a custom one with a servlet and some Arc beans.

Environment (please complete the following information):

• Output of uname -a or ver: Linux 8e5961cd5b12 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 13 10:46:25 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
• Output of java -version: openjdk version "1.8.0_222"
OpenJDK Runtime Environment (build 1.8.0_222-20190711120915.buildslave.jdk8u-src-tar--b08) OpenJDK 64-Bit GraalVM CE 19.2.0.1 (build 25.222-b08-jvmci-19.2-b02, mixed mode)
• GraalVM version (if different from Java): GraalVM CE 19.2.0.1
• Quarkus version or git rev: 0.23.1

Additional context

I've only seen the same error mentioned in apache/camel-quarkus#132

[lburgazzoli]

This can be reproduced using

https://github.com/lburgazzoli/apache-camel-quarkus/tree/quarkus-0.23.1/integration-tests/jdbc

[cescoffier]

Thanks @lburgazzoli, I'm on it.

[cescoffier]

@lburgazzoli Failed to build it:

[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:0.23.1:build (default) on project camel-quarkus-integration-test-jdbc: Failed to build a runnable JAR: Failed to build a runner jar: Failed to augment application classes: public void io.quarkus.undertow.deployment.UndertowBuildStep.registerUndertowHandlersConf(io.quarkus.deployment.annotations.BuildProducer,io.quarkus.deployment.builditem.ApplicationArchivesBuildItem,io.quarkus.deployment.annotations.BuildProducer,io.quarkus.deployment.annotations.BuildProducer) is marked @Record but does not inject an @Recorder object -> [Help 1]

Does that ring a bell?

[lburgazzoli]

It does not fail for me, did you install the whole project before ? That error should have been fixed

[cescoffier]

Issue reproduced! Going to work on a fix today

[cescoffier]

Ok, still struggling with this one.

Some research found that @FroMage introduced the Conscript deletion.

What I fail to understand if how it is triggered. Having the same set of dependency, in the same order does not trigger it.

[cescoffier]

The reportErrorsAtRuntime seems to be the cause

[lburgazzoli]

Yep, removing reportErrorsAtRuntime seems to make the test working, that's looks weird

[cescoffier]

working on a new set of substitutions working in both modes.