GH-37739: [Java] Add experimental arrow-memory-ffm module #38016

danepitkin · 2023-10-04T16:15:54Z

WIP

Rationale for this change

The FFM APIs had their 3rd preview release in Java 21. Let's start integrating them with Arrow to see how effective they are. The FFM APIs will presumably be finalized in the next Java LTS release (24).

JEP 442: https://openjdk.org/jeps/442
java.lang.foreign: https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/foreign/package-summary.html

What changes are included in this PR?

Add a new memory module based on java.lang.foreign.

Are these changes tested?

Unit tests added

Are there any user-facing changes?

Yes.

Closes: [Java] Implement arrow-memory-ffm arrow-java#163
GitHub Issue: ARROW-326: Initialize nested writers in MapWriter based on the underl… #163

github-actions · 2023-10-04T16:16:26Z

⚠️ GitHub issue apache/arrow-java#163 has been automatically assigned in GitHub to PR creator.

danepitkin · 2023-10-04T16:20:04Z

java/memory/memory-ffm/src/main/java/org/apache/arrow/memory/FfmAllocationManager.java

An Arena controls the lifecycle of native (off-heap) memory segments. The Shared Arena has a bounded lifetime, is explicitly closeable, and is accessible by multiple threads.[1]

[1]https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/foreign/Arena.html

pitrou · 2023-10-19T14:45:27Z

Java newbie here, can you give a pointer to the FFM docs?

danepitkin · 2023-10-19T15:14:59Z

Java newbie here, can you give a pointer to the FFM docs?

Added the JEP and docs to the description!

pitrou · 2023-10-19T16:27:53Z

Added the JEP and docs to the description!

Wow, that looks really promising.

What are the implications of using a preview API? Can the API change in later versions? Does it impact downstream users?

danepitkin · 2023-10-19T16:55:41Z

Wow, that looks really promising.

What are the implications of using a preview API? Can the API change in later versions? Does it impact downstream users?

Yes, the APIs can still change, but it is approaching stability now that its been in development for a few years. I plan to mark this module as experimental and only enabled for Java 21+, but I expect the JEP to be finalized for the next Arrow Java LTS (Long Term Support) release Java 24. The benefit of adding it now is that we can trial it and potentially provide late feedback to the JEP implementation if needed.

Adding the module itself won't impact users unless they choose to use it. Arrow Java provides an abstract interface to off-heap memory called arrow-memory-core. Users must then select exactly one implementation: sun.misc.Unsafe (arrow-memory-unsafe), netty (arrow-memory-netty), and now the experimental FFM (for Java 21+ only). The arrow-memory-core interface will need to eventually be refactored to take better advantage of FFM, but I don't plan to do that in this PR.

pitrou · 2023-10-19T16:58:49Z

Do you think we'll also ditch our JNI interfaces at some point? I suppose we must keep them until FFM becomes mainstream, but would it be beneficial to use FFM as an alternative on supported setups?

danepitkin · 2023-10-19T17:07:32Z

Do you think we'll also ditch our JNI interfaces at some point? I suppose we must keep them until FFM becomes mainstream, but would it be beneficial to use FFM as an alternative on supported setups?

Eventually, yes! I think there is a lot that needs to happen before then. Besides the Java support fiasco holding many users back (Java 8 has extended support until 2030) and needing to wait until Java 24+, the JIT implementations will also most likely require work to improve optimizations of FFM APIs to meet similar performance numbers as the existing alternatives. Long term, this definitely looks like the preferred approach over JNI to me.

danepitkin · 2023-11-13T20:41:39Z

java/memory/memory-ffm/src/main/java/org/apache/arrow/memory/FfmAllocationManager.java

+  FfmAllocationManager(BufferAllocator accountingAllocator, long requestedSize) {
+    super(accountingAllocator);
+    arena = Arena.ofShared();
+    allocatedMemorySegment = arena.allocate(requestedSize, /*byteAlignment*/ 8);


Note that FFM requires byteAlignment to be specified within the allocation manager, while netty/unsafe do not.

danepitkin · 2023-11-13T20:42:39Z

java/pom.xml

+        <jdk>[21,]</jdk>
+      </activation>
+      <modules>
+        <module>memory/memory-ffm</module>


Is this the best way to enable developers to build arrow-memory-ffm? I haven't come up with a better alternative, but I'm also not a POM expert.

CI does not build this module, but a developer can build from source with mvn clean install -pl :arrow-memory-ffm --am

danepitkin · 2023-11-13T20:51:50Z

@github-actions crossbow submit -g java

github-actions · 2023-11-13T20:54:22Z

Revision: 0cd7939

Submitted crossbow builds: ursacomputing/crossbow @ actions-0fc32fc5ee

Task	Status
java-jars
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64

adamkennedy · 2023-11-13T22:45:05Z

One tidbit passed on from the Trino team was that once we have FFI then we'd also want to figure out how to move past any uses of ByteBuffer in the rest of the code to the FFI classes, as (reportedly) some mistakes were made with ByteBuffer which means it isn't very friendly to inlining and the new stuff is much much faster once inlining has time to kick in.

adamkennedy · 2023-11-13T22:47:45Z

...memory/memory-core/src/main/java/org/apache/arrow/memory/DefaultAllocationManagerOption.java

+    try {
+      return getFactory("org.apache.arrow.memory.FfmAllocationManager");
+    } catch (RuntimeException e) {
+      if (majorVersion < 21) {


Would we also have to detect that preview features are actually turned on and the underlying FFM libraries are available in that specific Java 21+ VM instance?

Yes, good catch. More helpful checks would be nice.

docs/source/java/overview.rst

...memory/memory-core/src/main/java/org/apache/arrow/memory/DefaultAllocationManagerOption.java

danepitkin · 2023-11-13T22:58:23Z

One tidbit passed on from the Trino team was that once we have FFI then we'd also want to figure out how to move past any uses of ByteBuffer in the rest of the code to the FFI classes, as (reportedly) some mistakes were made with ByteBuffer which means it isn't very friendly to inlining and the new stuff is much much faster once inlining has time to kick in.

Thank you! I envision this refactor, plus supporting long memory addresses, as follow up tasks. I can file those if this PR merges without those features.

danepitkin · 2023-11-14T23:10:01Z

I'm going to put this on hold for now. The next steps are to refactor out the usage of UNSAFE since it's not accessible in Java 16+. This is a huge task, but would allow us to actually test the memory-core module with FFM APIs. This PR remains an experiment and is not in any mergable form.

danepitkin · 2023-11-14T23:12:14Z

java/memory/memory-core/src/test/java/org/apache/arrow/memory/JavaForeignAllocationManager.java

@@ -0,0 +1,77 @@
+/*


This file was added as a hack to test the memory-core module with the FFM APIs. It will be useful for testing later, but can be ignored otherwise.

wendigo · 2024-04-16T20:57:15Z

Since FFM is GA since JDK 22, is there a plan to move forward with FFM-based allocator for Arrow?

vibhatha · 2024-04-17T02:54:19Z

@wendigo Yes we are planning to.

danepitkin · 2024-09-09T16:01:24Z

Closing for now due to staleness.

github-actions · 2024-11-26T19:16:32Z

⚠️ GitHub issue #37739 has no components, please add labels for components.

danepitkin requested review from assignUser, kou, lidavidm and raulcd as code owners October 4, 2023 16:15

danepitkin marked this pull request as draft October 4, 2023 16:16

github-actions bot added Component: Java Component: Documentation awaiting review Awaiting review labels Oct 4, 2023

danepitkin commented Oct 4, 2023

View reviewed changes

github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Oct 4, 2023

danepitkin force-pushed the danepitkin/arrow-memory-ffm branch from bb22994 to c8b78bc Compare October 19, 2023 02:03

Dane Pitkin added 2 commits November 13, 2023 12:29

Add experimental arrow-memory-ffm module

ecae6f1

only allow explicit build for arrow-memory-ffm

609e7ef

danepitkin force-pushed the danepitkin/arrow-memory-ffm branch from c8b78bc to 609e7ef Compare November 13, 2023 20:00

Dane Pitkin added 3 commits November 13, 2023 15:04

Remove unecessary edits

1ca51fc

Clean up documentation wording

ef6665d

Add runtime exception message for version mismatch

0cd7939

danepitkin commented Nov 13, 2023

View reviewed changes

danepitkin marked this pull request as ready for review November 13, 2023 21:27

adamkennedy reviewed Nov 13, 2023

View reviewed changes

docs/source/java/overview.rst Outdated Show resolved Hide resolved

adamkennedy reviewed Nov 13, 2023

View reviewed changes

...memory/memory-core/src/main/java/org/apache/arrow/memory/DefaultAllocationManagerOption.java Outdated Show resolved Hide resolved

address comments, update maven profile

8c05046

danepitkin marked this pull request as draft November 14, 2023 23:06

danepitkin commented Nov 14, 2023

View reviewed changes

wendigo mentioned this pull request Apr 30, 2024

Add Trino 446 release notes trinodb/trino#21753

Merged

danepitkin closed this Sep 9, 2024

vibhatha mentioned this pull request Jan 15, 2025

Memory-Access Methods in sun.misc.Unsafe are deprecated for removal apache/arrow-java#511

Open

GH-37739: [Java] Add experimental arrow-memory-ffm module #38016

GH-37739: [Java] Add experimental arrow-memory-ffm module #38016

Uh oh!

Conversation

danepitkin commented Oct 4, 2023 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

WIP

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions bot commented Oct 4, 2023

Uh oh!

danepitkin Oct 4, 2023

Choose a reason for hiding this comment

Uh oh!

pitrou commented Oct 19, 2023

Uh oh!

danepitkin commented Oct 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pitrou commented Oct 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danepitkin commented Oct 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pitrou commented Oct 19, 2023

Uh oh!

danepitkin commented Oct 19, 2023

Uh oh!

danepitkin Nov 13, 2023

Choose a reason for hiding this comment

Uh oh!

danepitkin Nov 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danepitkin commented Nov 13, 2023

Uh oh!

github-actions bot commented Nov 13, 2023

Uh oh!

adamkennedy commented Nov 13, 2023

Uh oh!

adamkennedy Nov 13, 2023

Choose a reason for hiding this comment

Uh oh!

danepitkin Nov 13, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

danepitkin commented Nov 13, 2023

Uh oh!

danepitkin commented Nov 14, 2023

Uh oh!

danepitkin Nov 14, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wendigo commented Apr 16, 2024

Uh oh!

vibhatha commented Apr 17, 2024

Uh oh!

danepitkin commented Sep 9, 2024

Uh oh!

github-actions bot commented Nov 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

danepitkin commented Oct 4, 2023 •

edited by github-actions bot

Loading

danepitkin commented Oct 19, 2023 •

edited

Loading

pitrou commented Oct 19, 2023 •

edited

Loading

danepitkin commented Oct 19, 2023 •

edited

Loading

danepitkin Nov 13, 2023 •

edited

Loading

danepitkin Nov 14, 2023 •

edited

Loading