Share getThreadManager pool among RuntimeServerInstrument and Websocket emulation

[JaroslavTulach]

Pull Request Description

Resolves #11477 by making callbacks from ydoc-polyfill WebSocket emulation via the same ScheduledExecutorService as provided by EnsoLanguage. In addition to that it registers threadAccessDeniedHandler to coordinate JavaScript multi threaded execution.

Important Notes

The EpbLanguage (which injects the WebSocket emulation) locates the EnsoLanguage by its ID and obtains a reference to EnsoContext.getThreadManager() via lookup(langInfo, ScheduledExecutorService.class). Then it uses the obtained executor service for making callbacks into JavaScript whenever a websocket event arrives.

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

• All code follows the
  Scala,
  Java,
• Manual testing in the IDE works

[JaroslavTulach]

Manual Testing

Let's use a simple script that connects to echo service at http://websocket.org to verify how well the support for WebSocket API works in Enso IDE:

var counter = 0;
function app() {
    print("Connecting to websocket.org...");
    let ws  = new WebSocket('wss://echo.websocket.org');
    function sayHi() {
      ws.send(`Hi there! #${counter++}`);
    }
    ws.onopen = function(ev) {
      print("Open: " + ev);
    };
    ws.onmessage = function(ev) {
      print("Msg   : " + Object.getOwnPropertyNames(ev));
      print("  data: " + ev.data);
      setTimeout(sayHi, 3000);
    };
    ws.onclose = function(ev) {
      print("Close: " + Object.getOwnPropertyNames(ev));
    };
    ws.onerror = function(ev) {
      print("Err: " + Object.getOwnPropertyNames(ev));
    };
}

if (typeof insight === "undefined") {
    globalThis.print = console.log;
    if (typeof WebSocket === "undefined") {
        let r = require("ws");
        globalThis.WebSocket = r.WebSocket;
    }
    app();
} else {
    app();
}

The script is generic. You can use it in node.js as well as Enso IDE or command line interface.

Enso GUI

Store the above script as /tmp/wstest.js and then execute following commands in the root of Enso repository:

enso$ export ENSO_JVM_OPTS="-Denso.dev.insight=/tmp/wstest.js -Dpolyglot.enso.interpreter.jobParallelism=1"
enso$ sbt runProjectManagerDistribution
....
Connecting to websocket.org...
...
Msg   : 
  data: Hi there! #2
...

and in another terminal execute corepack pnpm dev:gui. Connect your browser to http://localhost:5173/, open a project and observe messages printed by the sbt runProjectManagerDistribution. The same output as in case of node.js (see below) shall appear.

Verify with node.js

Save the above script into an empty directory ws as wstest.js and execute:

ws$ npm install ws
ws$ node wstest.js 
Connecting to websocket.org...
Open: [object Event]
Msg   : 
  data: Request served by 1781505b56ee58
Msg   : 
  data: Hi there! #0
Msg   : 
  data: Hi there! #1
Msg   : 
  data: Hi there! #2

and so on. Every few seconds another "Hi there!" message with an increased counter shall appear. The same output shall be visible in the Enso GUI as well as from CLI.

[JaroslavTulach]

@hubertp, @4e6, do you think following assumption:

the assumption that all Enso code is executing in the pool controlled by the jobParallelism property

Is supposed to be true? E.g. if I fix the violations and set jobParallelism=1 - will JavaScript callbacks be properly serialized one and another and intermixed with regular requests to execute Enso code?

Update on Apr 10, 2025:

• since Upgrading to Truffle libraries 24.2.0 #12500 there is GUEST_PARALLELISM option
• by default it is currently one
• but it can be used to increase the guest executor thread throughput
• since 343f3b9 the EnsoContext.getThreadManager() is an ScheduledExecutorService
• the service is also used by EpbLanguage emulation - https://github.com/enso-org/enso/pull/12528/files#r2036474840
• since 0e03731 - we are using threadAccessDeniedHandler to coordinate JavaScript execution even in multi threaded pool

[JaroslavTulach]

We seem to have problems with shutdown:

The language did not complete all polyglot threads but should have: [Thread[#546,guest-code-1,5,main]]

probably the shutdown isn't done properly.

[JaroslavTulach]

b5b430a avoids usage of TruffleLogger from non-guest code threads.

[hubertp]

OK

[hubertp]

shouldn't we log this event always?

[JaroslavTulach]

• I was thinking about some heuristics to decide when to log, but
• at the end I decided to leave it out of this PR
• I am not sure logging (and dumping all the thread stacks) always is what we want
• there might be plenty of events like this
• a one line note in the log when verbosity is high might be OK
• so far I haven't seen many clashes even I am running with -ea on

[JaroslavTulach]

Let's merge and continue with

• Deliver expression updates via Ydoc AST elements #11159
• and Execute yjs from insight script #11965