Java Loader

[ketangupta34]

Implementation of Java Loader

This is the issue for the discussion related to the development of the Java Loader.

Proposed Solution

1) Develop the PoC for the proposed idea.

• Implement the bootstrap class implementing some major functions related to the java loader, which include callFunction, LoadFromFile, LoadFromMemory, etc, using the java Compiler API.
• Implement the cpp functions that call the bootstrap function using the java Native Interface.

PoC tested with java 8, java 11, gcc 10, and clang 11.

2) Implementing the metacall core

• Map the functions in PoC to function required in metacall core and create the list of additional functions which are required for the proper functioning of the project. (In Progress)
• Start the initial implementation of java Loader

Proof of Concept

The proof of concept done by @ketangupta34 can be found here -
https://github.com/ketangupta34/JNI/tree/jni-compiler-PoC

To run and compile the PoC -

g++ Caller.cpp \
-I"/usr/lib/jvm/java-11-openjdk-amd64/include" \
-I"/usr/lib/jvm/java-11-openjdk-amd64/include/linux" \
-L"/usr/bin/java" \
-L"/usr/lib/jvm/java-11-openjdk-amd64/lib/server" -ljvm

Above command can be run using java 8+ by replace the version in command to the java version available on your machine.

[viferga]

I am going to add a list of TODO tasks, so the whole project can be break into small pieces. Feel free to edit the original post with them (and then I can delete this one).

• Initialize JVM without a hardcoded classpath (this will cause a recursive dependency, we cannot add a classpath without initializing the JVM and the JVM needs the classpath to be initialized, probably we have to find a way to initialize the classpath dynamically through the following task):

  core/source/loaders/java_loader/source/java_loader_impl.cpp

  Line 117 in a72d15c

  loader_impl_data java_loader_impl_initialize(loader_impl impl, configuration config)

• Add a new execution path (basically, add a new path to the classpath):

  core/source/loaders/java_loader/source/java_loader_impl.cpp

  Line 162 in a72d15c

  int java_loader_impl_execution_path(loader_impl impl, const loader_naming_path path)

• Implement load from file (load a java file).
• Implement load from memory (eval).
• Implement load from package (load a jar).
• Implement type initialization.
• Implement introspection with class support (I think Java does not support top level functions, right? Everything must be wrapped into a class, so probably we wont need to support classes).
• Make the test metacall_java_test work properly with the loader and the classes (now the test is wrong, it is designed as it was dealing with static methods from classes as functions, which is how C# works now, but that's an old design, right now we support classes so the ideal would be to replace it by using classes properly and do TDD).
• Implement clear command, it should be able to remove all the compiled classes from the JVM in memory, not sure if Java supports this but it should clear all resources used by the loaded code.
• Destroy the JVM and all the resources:

  core/source/loaders/java_loader/source/java_loader_impl.cpp

  Line 294 in a72d15c

  int java_loader_impl_destroy(loader_impl impl)

• Remove std::cout data and use log_write instead.
• Automatize the jar projects with CMake for load from package tests.

[viferga]

Here's the updated version of the test: https://github.com/metacall/core/tree/develop/source/tests/metacall_java_test
Here's the script that is being loaded: https://github.com/metacall/core/tree/develop/source/scripts/java/fibonacci/source