ST6RI-766 Update to JupyterLab version 4.x in the Jupyter deployment

[TheKorpos]

This PR includes dependency and code updates for the org.omg.sysml.jupyter.jupyterlab labextension to support Jupyter 4.x. Older versions are no longer supported.

This PR doesn't change how the extension is packaged. Installing it into Jupyter is still possible with jupyter labextension install (which is deprecated but still works in 4.x).

Dependency updates

• updated dependencies in package.json
  • "@jupyterlab/application": "3.x" -> "4.x"
  • "typescript": "<4.4.0" -> "~5.8.3"
  • "@jupyterlab/codemirror": "^0.0.98" -> ">=4.0"
• added new dependency
  • "@codemirror/legacy-modes": "^6.3.2"
• updated compiler options
  • es2018 -> es2023

Code changes

Previously, the syntax highlight was registered by adding a new "mode" to Codemirror. This is no longer an option. Instead, the language has to be registered via IEditorLanguageRegistry by the front-end plugin. Luckily, the underlying parser is still present and can be registered in "legacy" mode, so most of the configuration can be easily ported.

• updated the implementation of the front-end plugin
  • language registry is used to register the sysml language
• instead of CodeMirror mode, a clike parser is created
  • reused (most of) the configuration from older code

[seidewitz]

I noticed that the new extension does not support folding of code between braces. That is, the old mode definition included:

modeProps: {
    fold: ['brace'],
}

but I don't see anything similar in the new sysmlparser definition. If there is no way to do this in the new approach, that is a serious degradation of functionality that I know is commonly used by users.

[seidewitz]

Also, the file org.omg.sysml.jupyter.kernel/src/main/resources/sysml/kernel.js still sets up a CodeMirror mode for classic Jupyter Notebook. Will this still work? Do we even need to support classic Jupyter Notebook anymore?

[TheKorpos]

Also, the file org.omg.sysml.jupyter.kernel/src/main/resources/sysml/kernel.js still sets up a CodeMirror mode for classic Jupyter Notebook. Will this still work? Do we even need to support classic Jupyter Notebook anymore?

No, this shouldn't work. I always go for jupyter lab. I'm not sure if there is a high demand for classic Notebook.

[TheKorpos]

I noticed that the new extension does not support folding of code between braces. That is, the old mode definition included:

modeProps: {
    fold: ['brace'],
}

but I don't see anything similar in the new sysmlparser definition. If there is no way to do this in the new approach, that is a serious degradation of functionality that I know is commonly used by users.

I've tried adding code folding as an extension in the frontend plugin but I couldn't get it working unfortunately. I couldn't really find any option either that would indicate some easy turn-on.

import {codeFolding, foldGutter} from '@codemirror/language' I tried using these. These seem to create the extensions for the folding. I tried to register these with IEditorExtensionRegistry but it didn't work...

[himi]

I think we need to use foldService for CodeMirror V6.

[seidewitz]

Also, the file org.omg.sysml.jupyter.kernel/src/main/resources/sysml/kernel.js still sets up a CodeMirror mode for classic Jupyter Notebook. Will this still work? Do we even need to support classic Jupyter Notebook anymore?

No, this shouldn't work. I always go for jupyter lab. I'm not sure if there is a high demand for classic Notebook.

If it is not going to work anyway, we should take it out of the build.

[seidewitz]

I think we need to use foldService for CodeMirror V6.

The problem is that the v6 foldService doesn't work for legacy language support, because it requires real parsing of the language text. As far as I could determine, it is possible to get legacy code folding to work, but one needs to write a routine to determine the scope of the folding (e.g., based on { ... } delimiters). Surprisingly, I couldn't find anything in CodeMirror or otherwise publicly available to do this. I actually got what looked like a pretty good code suggestion from AI (even though using AI for code is against my usual principles!), so I might try seeing if that can work.

In any case, if we can't find a way to get code folding incorporated fairly quickly, I am going to defer this PR to the next sprint. I don't want to deploy it without code folding.

[himi]

We can use foldService with @codemirror/legacy-modes/mode but I do not know how we enable CodeMirror 6 extensions in legacy JupyterLab extensions. (As I said, I already switched to new JupyterLab, Python Wheel-based extension) Anyway, it would not be a simple task.

[seidewitz]

I am going to remove this PR from the release. We figure out code folding on the next sprint.