Skip to content

Conversation

@Akirathan
Copy link
Member

@Akirathan Akirathan commented Sep 17, 2025

Closes #13918

Pull Request Description

Update GraalVM from 24.0.1 to 25.0.0.

Important Notes

Highlights from GraalVM 25 release notes:

  • FFM API support on aarch64
  • Implemented Arena.ofShared
  • Experimental Native image tracing agent
  • Experimental -H:+JDWP flag.

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

  • The documentation has been updated, if necessary.
  • Screenshots/screencasts have been attached, if there are any visual changes. For interactive or animated visual changes, a screencast is preferred.
  • All code follows the
    Scala,
    Java,
    TypeScript,
    and
    Rust
    style guides. In case you are using a language not listed above, follow the Rust style guide.
  • Unit tests have been written where possible.
  • If meaningful changes were made to logic or tests affecting Enso Cloud integration in the libraries,
    or the Snowflake database integration, a run of the Extra Tests has been scheduled.
    • If applicable, it is suggested to paste a link to a successful run of the Extra Tests.

@Akirathan Akirathan self-assigned this Sep 17, 2025
@Akirathan Akirathan added the CI: Clean build required CI runners will be cleaned before and after this PR is built. label Sep 17, 2025
@Akirathan
Copy link
Member Author

NI build of engine-runner is successful. Its size is 533 MB. The previous size (for example built here) was 568 MB.

So just by upgrading to GraalVM 25.0.0, the NI size decreased by roughly 6%.

@Akirathan
Copy link
Member Author

Akirathan commented Sep 22, 2025

Failure in std-table/test: Mockito failure: "Java 25 not supported" at https://github.com/enso-org/enso/actions/runs/17914034971/job/50933411374?pr=14019#step:14:2018.

Fixed by bumping Mockito version in 6ac2d05

mergify bot pushed a commit that referenced this pull request Sep 23, 2025
#14019 is blocked by this - the current google java format version is failing because it tries to call remove method from JDK - see https://github.com/enso-org/enso/actions/runs/17916525266/job/50940077628?pr=14019#step:5:245

# Important Notes
The only notable change is in ff3fec5 . Every thing else is just automatic `sbt javafmtAll` formatting.
@Akirathan Akirathan marked this pull request as ready for review September 23, 2025 17:07
@enso-bot enso-bot bot mentioned this pull request Oct 10, 2025
3 tasks
@Akirathan
Copy link
Member Author

Native stdlib Windows tests are still failing on NPE in Table_Tests. I suppose this is because classes from Table_Tests are loaded by HotspotVM? cc @JaroslavTulach. Any suggestions are welcomed.

@JaroslavTulach
Copy link
Member

Native stdlib Windows tests are still failing on NPE in Table_Tests. I suppose this is because classes from Table_Tests are loaded by HotspotVM? cc @JaroslavTulach. Any suggestions are welcomed.

  • I don't think Table_Tests should be using dual JVM mode
  • but Generic_JDBC_Tests do use it, solution:
    • modify Rust build script to pass --jvm on Windows when running test/Generic_JDBC_Tests
    • or return back this line - which would switch to JVM mode on all platforms

@JaroslavTulach
Copy link
Member

There is a linking difference between older GraalVMs and new GraalVM 25. The DLL is linked to wrong symbols.

@JaroslavTulach
Copy link
Member

JaroslavTulach commented Oct 17, 2025

This PR is currently blocked by oracle/graal#12384 - the problem is /EXPORT:Java_sun_nio_fs_WindowsNativeDispatcher_FindFirstFile0 from nio.lib: oracle/graal#12384 (comment)

The solution is probably described at this Stackoverflow inquiry:

  • create empty .def file
  • Create an .exp file from the .def file using LIB
  • Link the executable with this .exp file
  • there is a way to specify any options to native-image - e.g. just modify build.sbt to link with empty exp file

@JaroslavTulach
Copy link
Member

  • This PR is currently blocked
  • the problem is /EXPORT:Java_sun_nio_fs_WindowsNativeDispatcher_FindFirstFile0 from nio.lib: oracle/graal#12384 (comment)

Copy link
Member

@JaroslavTulach JaroslavTulach Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Necessary for d4fc41c
  • the enso.exp file was generated on Windows by following command:
$ "/c/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/lib.exe" /def:enso.def

and uses the enso.def file integrated in d4fc41c as well. The same command also generates enso.lib.

build.sbt Outdated
val linkOpts = if (Platform.isWindows) {
val ensoExp = file("enso.exp")
Seq("-H:NativeLinkerOption=" + ensoExp)
Seq("-H:NativeLinkerOption=" + ensoExp.getAbsolutePath)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the 84bf379 change we can almost generate the enso.exe, but

  sbt ℹ️ [3/8] Building universe...                                                                             (59.2s @ 12.14GB)
  sbt ℹ️ [4/8] Parsing methods...      [***]                                                                     (8.7s @ 11.69GB)
  sbt ℹ️ [5/8] Inlining methods...     [****]                                                                     (9.2s @ 9.90GB)
  sbt ℹ️ [6/8] Compiling methods...    [******]                                                                 (36.2s @ 12.02GB)
  sbt ℹ️ [7/8] Laying out methods...   [*******]                                                                (55.9s @ 14.32GB)
  sbt ℹ️ 
  sbt ℹ️ [8/8] Creating image...       [******]                                                                  (0.0s @ 12.59GB)
  sbt ℹ️ ------------------------------------------------------------------------------------------------------------------------
  sbt ℹ️                        64.0s (8.9% of total time) in 511 GCs | Peak RSS: 19.11GB | CPU load: 7.98
  sbt ℹ️ ------------------------------------------------------------------------------------------------------------------------
  sbt ℹ️ Build artifacts:
  sbt ℹ️  C:\runner\_work\enso\enso\built-distribution\enso-engine-2025.3.1-dev-windows-amd64\enso-2025.3.1-dev\bin\awt.dll (jdk_library)
  sbt ℹ️  C:\runner\_work\enso\enso\built-distribution\enso-engine-2025.3.1-dev-windows-amd64\enso-2025.3.1-dev\bin\enso.exe (executable)
  sbt ℹ️  C:\runner\_work\enso\enso\built-distribution\enso-engine-2025.3.1-dev-windows-amd64\enso-2025.3.1-dev\bin\management_ext.dll (jdk_library)
  sbt ℹ️  C:\runner\_work\enso\enso\built-distribution\enso-engine-2025.3.1-dev-windows-amd64\enso-2025.3.1-dev\bin\svm_err_b_20251024T224807.988_pid5648.md (build_info)
  sbt ℹ️  C:\runner\_work\enso\enso\built-distribution\enso-engine-2025.3.1-dev-windows-amd64\enso-2025.3.1-dev\bin\w2k_lsa_auth.dll (jdk_library)
  sbt ℹ️ ========================================================================================================================
  sbt ℹ️ Failed generating 'enso' after 11m 56s.
  sbt ℹ️ 
  sbt ℹ️ The build process encountered an unexpected error:
  sbt ℹ️ 
  sbt ℹ️ com.oracle.svm.core.util.VMError$HostedError: should not reach here: unexpected input could not be handled: 2
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.VMError.shouldNotReachHereUnexpectedInput(VMError.java:97)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.jdk.JNIRegistrationSupport.makeShimLibrary(JNIRegistrationSupport.java:354)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.jdk.JNIRegistrationSupport.makeShimLibraries(JNIRegistrationSupport.java:312)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.jdk.JNIRegistrationSupport.afterImageWrite(JNIRegistrationSupport.java:252)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$doRun$8(NativeImageGenerator.java:752)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:93)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:752)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:535)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:548)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:761)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:163)
  sbt ℹ️ 	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:111)

maybe we have too little symbols this time.

Copy link
Member

@JaroslavTulach JaroslavTulach Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failure happens when invoking link.exe comamnd. With 4928295 we should see what are the exact parameters and from that we should be able to deduce symbols that are missing. Some debug output is already visible.

The symbols exported from enso.exe by GraalVM 24 are:

/cygdrive/c/Program\ Files/Microsoft\ Visual\ Studio/2022/Community/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/link.exe /DUMP ./built-distribution/enso-engine-0.0.0-dev-windows-amd64/enso-0.0.0-dev/bin/enso.exe /exports 

Microsoft (R) COFF/PE Dumper Version 14.44.35215.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file .\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\bin\enso.exe

File Type: EXECUTABLE IMAGE

  Section contains the following exports for enso.exe

    00000000 characteristics
    FFFFFFFF time date stamp
        0.00 version
           1 ordinal base
          61 number of functions
          61 number of names

    ordinal hint RVA      name

          1    0 005F1070 Channel__acceptRequestFromHotSpotJvm__xqLwEJ57e96Rqd6MeQ6rTG
          2    1 005F0160 IsolateEnterStub__CEntryPointNativeFunctions__attachThread__LWiahz8fydJWgxqffuaEC2
          3    2 005F0340 IsolateEnterStub__CEntryPointNativeFunctions__createIsolate__aVxZJEvxfd8FslyQXMlHNI
          4    3 005F0550 IsolateEnterStub__CEntryPointNativeFunctions__detachAllThreadsAndTearDownIsolate__OPBluFOA7k6a9G1cqb6TTL
          5    4 005F07B0 IsolateEnterStub__CEntryPointNativeFunctions__detachThread__vZogK8TBGAIeNIWWaZu2QH
          6    5 005F0B00 IsolateEnterStub__CEntryPointNativeFunctions__getCurrentThread__p19YkdKU3I3cgUmQuiiTd0
          7    6 005F0CF0 IsolateEnterStub__CEntryPointNativeFunctions__getIsolate__4JMPcmfHydBLuvj7ff2do8
          8    7 005F0D20 IsolateEnterStub__CEntryPointNativeFunctions__tearDownIsolate__I3DVyDKs6z4vL79kJC9BEE
          9    8 005F1070 IsolateEnterStub__Channel__acceptRequestFromHotSpotJvm__s16uPKjf7yIi1jeDkTCti8
         10    9 005F13C0 IsolateEnterStub__InternalResourceCacheSymbol__internalResourceCacheSymbol__nxyn1Pb8T5JGPAbLqZZoU1
         11    A 00627EE0 IsolateEnterStub__JNIInvocationInterface_0024Exports__JNI__CreateJavaVM__zfv8XhXAFhERNPUoh38uvF
         12    B 00628250 IsolateEnterStub__JNIInvocationInterface_0024Exports__JNI__GetCreatedJavaVMs__jTTCq2UmbC48XDHPdLhle5
         13    C 00628470 IsolateEnterStub__JNIInvocationInterface_0024Exports__JNI__GetDefaultJavaVMInitArgs__8P9gtUmW2O2BqcAmkyOH99
         14    D 00629B30 IsolateEnterStub__JavaMainWrapper__run__cXbfAhOWcF90761nQYco7L
         15    E 0062A110 IsolateEnterStub__VmLocatorSymbol__vmLocatorSymbol__8qpvpOcqTdCFagkhQGXtTD
         16    F 046A7ECC JDK_LoadSystemLibrary
         17   10 00627EE0 JNI_CreateJavaVM
         18   11 00628250 JNI_GetCreatedJavaVMs
         19   12 00628470 JNI_GetDefaultJavaVMInitArgs
         20   13 046963A0 JNU_CallMethodByName
         21   14 046963C4 JNU_CallMethodByNameV
         22   15 0469660C JNU_CallStaticMethodByName
         23   16 04696800 JNU_ClassString
         24   17 04696920 JNU_GetEnv
         25   18 0469693C JNU_GetFieldByName
         26   19 04696B10 JNU_GetStaticFieldByName
         27   1A 04696CE4 JNU_IsInstanceOfByName
         28   1B 04696D6C JNU_NewObjectByName
         29   1C 04696E04 JNU_NewStringPlatform
         30   1D 04696EC4 JNU_SetFieldByName
         31   1E 04697080 JNU_ThrowArrayIndexOutOfBoundsException
         32   1F 04697090 JNU_ThrowByName
         33   20 046972D4 JNU_ThrowIOException
         34   21 046972F4 JNU_ThrowIllegalArgumentException
         35   22 04697304 JNU_ThrowInternalError
         36   23 04697314 JNU_ThrowNullPointerException
         37   24 04697324 JNU_ThrowOutOfMemoryError
         38   25 046A6690 JVM_CurrentTimeMillis
         39   26 046910A0 JVM_IsStaticallyLinked
         40   27 046A6790 JVM_RaiseSignal
         41   28 00001000 __svm_code_section
         42   29 0478BF20 __svm_version_info
         43   2A 0478BEF0 __svm_vm_java_version
         44   2B 0478C078 __svm_vm_target_ccompiler
         45   2C 0478C0C0 __svm_vm_target_libc
         46   2D 0478C250 __svm_vm_target_libraries
         47   2E 0478C1F8 __svm_vm_target_platform
         48   2F 0478C428 __svm_vm_target_staticlibraries
         49   30 046A81A0 getEncodingFromLangID
         50   31 046A83BC getJavaIDFromLangID
         51   32 005F0160 graal_attach_thread
         52   33 005F0340 graal_create_isolate
         53   34 005F0550 graal_detach_all_threads_and_tear_down_isolate
         54   35 005F07B0 graal_detach_thread
         55   36 005F0B00 graal_get_current_thread
         56   37 005F0CF0 graal_get_isolate
         57   38 005F13C0 graal_resource_cache_symbol
         58   39 005F0D20 graal_tear_down_isolate
         59   3A 0062A110 graal_vm_locator_symbol
         60   3B 046A6A70 jio_snprintf
         61   3C 00629B30 main

  Summary

        4000 .data
      140000 .pdata
       E2000 .rdata
       AD000 .reloc
        F000 .rodata
     73B0000 .svm_hea
     46A8000 .text

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • This is the last failure
  • let's try how that line looks without the special linker options

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reproduced the problem locally. The failure is

># LINK : warning LNK4077: exports file 'C:\Users\jst\NetBeansProjects\enso\enso.exp' used; ignoring other export specifications
>> 'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.44.35207\bin\HostX64\x64\cl.exe' '/FeC:\Users\jst\NetBeansProjects\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\bin\java.dll' 'C:\Users\jst\NetBeansProjects\enso\SVM-1761415826519\enso.lib' msvcrt.lib /link /dll /implib:java.lib /export:JDK_LoadSystemLibrary /export:JNU_CallMethodByName /export:JNU_CallMethodByNameV /export:JNU_CallStaticMethodByName /export:JNU_ClassString /export:JNU_GetEnv /export:JNU_GetFieldByName /export:JNU_GetStaticFieldByName /export:JNU_IsInstanceOfByName /export:JNU_NewObjectByName /export:JNU_NewStringPlatform /export:JNU_SetFieldByName /export:JNU_ThrowArrayIndexOutOfBoundsException /export:JNU_ThrowByName /export:JNU_ThrowIOException /export:JNU_ThrowIllegalArgumentException /export:JNU_ThrowInternalError /export:JNU_ThrowNullPointerException /export:JNU_ThrowOutOfMemoryError /export:getEncodingFromLangID /export:getJavaIDFromLangID  
># Microsoft (R) C/C++ Optimizing Compiler Version 19.44.35215 for x64
># Copyright (C) Microsoft Corporation.  All rights reserved.
>#
># Microsoft (R) Incremental Linker Version 14.44.35215.0
># Copyright (C) Microsoft Corporation.  All rights reserved.
>#
># /out:C:\Users\jst\NetBeansProjects\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\bin\java.dll
># /dll
># /implib:java.lib
># /export:JDK_LoadSystemLibrary
># /export:JNU_CallMethodByName
># /export:JNU_CallMethodByNameV
># /export:JNU_CallStaticMethodByName
># /export:JNU_ClassString
># /export:JNU_GetEnv
># /export:JNU_GetFieldByName
># /export:JNU_GetStaticFieldByName
># /export:JNU_IsInstanceOfByName
># /export:JNU_NewObjectByName
># /export:JNU_NewStringPlatform
># /export:JNU_SetFieldByName
># /export:JNU_ThrowArrayIndexOutOfBoundsException
># /export:JNU_ThrowByName
># /export:JNU_ThrowIOException
># /export:JNU_ThrowIllegalArgumentException
># /export:JNU_ThrowInternalError
># /export:JNU_ThrowNullPointerException
># /export:JNU_ThrowOutOfMemoryError
># /export:getEncodingFromLangID
># /export:getJavaIDFromLangID
># C:\Users\jst\NetBeansProjects\enso\SVM-1761415826519\enso.lib
># msvcrt.lib
># LINK : fatal error LNK1181: cannot open input file 'C:\Users\jst\NetBeansProjects\enso\SVM-1761415826519\enso.lib'

because there is no enso.lib in the temporary directory. It is not created there probably because we provide enso.exp file manully. The lib.exe command then also generates enso.lib, but not in the temporary directory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the help of debugger one can stop the execution at

debugger

move the enso.lib file into the temporary directory and re-execute the frame again. Then the enso.exe is built and all tests are passing:

\enso> .\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\bin\enso.exe --run test/Base_Tests
2574 tests succeeded.
93 tests skipped.
22 groups skipped.

the remaining question is: how to achieve the same without a debugger?

Copy link
Member

@JaroslavTulach JaroslavTulach Oct 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With da85c08 I am able to successfully execute test/Base_Tests. Notes to GraalVM guys provided at oracle/graal#12384 (comment)

@JaroslavTulach JaroslavTulach self-requested a review October 26, 2025 20:13
Copy link
Member

@JaroslavTulach JaroslavTulach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All engine CI tasks are green, @Akirathan please review my changes. There are some project service failures - do you know what they can be caused by, @4e6?

@JaroslavTulach JaroslavTulach added the CI: Keep up to date Automatically update this PR to the latest develop. label Oct 27, 2025
boolean found = false;
try {
var from = new File("enso.lib").getAbsoluteFile();
System.err.println("Distributing enso.lib to (temporary) directories. From " + from);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left over? I don't think you want to keep writing to stderr

Copy link
Collaborator

@hubertp hubertp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enso.{def,exp,lib} files must not be in the top directory. It's a packaging thing so probably somewhere in the distribution/resources directory?
Instructions to generate them should also be added either as a comment in the files (probably not possible) or in some docs related to distribution. PR comment is not really enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI: Clean build required CI runners will be cleaned before and after this PR is built. CI: Keep up to date Automatically update this PR to the latest develop.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upgrade to GraalVM 25 LTS

4 participants