Gradle Plugin Integration Test should, ideally, actually run the application

[vorburger]

While #1405 (#1396) will help with future non-regression for Gradle related pure build issues such as e.g. #1323, it does not really e2e test the Gradle built artifact, and thus misses problems such as #1622 and other possible future ones like it.

In fact, we could complete break the Gradle plugin and produce an empty 0 bytes JAR, as long as the Gradle build process itself does not fail, and the current Gradle IT as is would still pass... 😈

To add real coverage to actually start and do an HTTP GET on the Gradle built JAR does not seem terribly difficult, but perhaps one point to first discuss here to agree on what the best way to go about to launch (the equivalent) of java -jar build/getting-started-runner.jar from a JUnit is? Do we already do something like this elsewhere in Quarkus (perhaps Maven) ITs which could could & should be used as inspiration? Otherwise I would propose to use (my) https://github.com/vorburger/ch.vorburger.exec, but I thought it was worth first discussing this here before possibly raising a PR...

@stalep @jnizet @geoand any input, or even contribution, certainly welcome.

[geoand]

@vorburger I did both a (Maven) build and a java -jar ... as part of one of my open PRs. See this

[stalep]

Hi, the tests annotated with @QuarkusTest will not run the -runner.jar, but the compile classes, this is identical for both Maven and Gradle.
The @substratetest tests will actually start the native image as a process and test against it.
The "problem" atm with how Gradle differs from Maven is that Gradle will try to run all the tests under src/test while Maven will run tests ending with "IT.java" after the native image have been built.
For Gradle the integration tests should be moved to eg src/integTest and only be run after the quarkusNative task have completed.

[vorburger]

@stalep I was actually more thinking about an IT for the Gradle built JAR, but you are raising a good point that ideally such a new Gradle IT should cover both the runner JAR as well as a Gradle built native executable, yes.

[vorburger]

I gave this some thought today, and wanted to propose a PR with something simple and pragmatic as a starting point (just via a gradle-it-run kind of thing, which actually runs the Gradle built JAR...) - but then noticed that the quarkus/devtools/gradle-it is actually broken on current 1964a5d master (it used to work before; which underlines the importance of adding this IT for future non-regression), see:

cd quarkus/devtools/gradle-it
rm -rf build/ ; ../../mvnw clean install
java -jar build/gradle-it-runner.jar
Exception in thread "main" java.lang.NoClassDefFoundError: io/quarkus/runtime/Application
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
Caused by: java.lang.ClassNotFoundException: io.quarkus.runtime.Application
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 13 more

But given that the Gradle Quick Start still works (even after quarkusio/quarkus-quickstarts#137 and with quarkusio/quarkus-quickstarts@master...vorburger:gradle_v999-SNAPSHOT), what's "broken" here is likely "just" the quarkus/devtools/gradle-it POM - is it missing some dependency where io.quarkus.runtime.Application is? @geoand @Sanne @aloubyansky @dmlloyd would you happen to know what could cause this ClassNotFoundException: io.quarkus.runtime.Application?

PS: Also note #1926, but that's more orthogonal (it does not fix this).

[dmlloyd]

The missing dependency would be quarkus-core. So something is definitely seriously wrong. What platform is this?

[vorburger]

The missing dependency would be quarkus-core.

Thanks! But I do see quarkus-core-999-SNAPSHOT.jar in target/dependencies/compile/ ...

So something is definitely seriously wrong.

FYI note that quarkus/devtools/gradle-it/pom.xml uses maven-dependency-plugin to populate target/dependencies/compile/, which quarkus/devtools/gradle-it/build.gradle then refers to using compileOnly fileTree(dir: 'target/dependencies/compile', include: '*.jar') ...

But given that the Gradle Quick Start still works

... so this (above) technically strictly speaking is not exactly the same way (layout on the file system) of getting dependencies. It should not matter, but if the code which the Gradle plugin uses (which I believe is shared with the Maven plugin, and where @aloubyansky recently did some work to improve things?) somehow relies on finding things laid out in Maven local repo layout, perhaps that's causing this.. Let me try if I can change the IT to be more like IRL (like quickstart) - kind of like #1926 but pushing ever further in that direction, to align IT with actual use.

What platform is this?

I'm hitting this on Linux.

[vorburger]

Let me try if I can change the IT to be more like IRL (like quickstart) - kind of like #1926 but pushing ever further in that direction, to align IT with actual use.

Ha! To my surprise, that actually helped... see #1929 (specifically its 6f4ba31). With #1929, I now get a working JAR built by Gradle in quarkus/devtools/gradle-it again!

But now the tests fail under Gradle (so to get the JAR you actually need to build it using ../gradle/gradlew -x test build to skip tests), but that may be an entirely different problem - the basic quickstart, with Gradle support, does not actually have tests, so may be that also just broke recently? I'll have a closer look.