How renderers handle extension code activation

[rebornix]

Notebook renderers are usually contributed statically through contributions in package.json but we also allow communication between extension code and the renderers in the webview, through webview communication API. Having the ability to talk to the extension means that the renderer code might not be able to handle output rendering without some sort of setup in the extension host side, but this would degrade the file open experience as we don't want all renderers extensions to be activated prior to ipynb file opening.

To ensure a smooth file opening experience, VS Code will only wait for the serializer to be activated. This means extensions which contribute renderers will need to handle the activation/evaluation of extension code and renderer code properly.

[rchiodo]

Maybe have some sort of global event in the window that renderers can listen for? Like 'extension-loaded' or something?

[DonJayamanne]

Summary of discussion (recommended solution is Solution 3:

Problem:

• Assume we have two extensions, renderer Extension R & and regular Extension A.
• When a notebook is opened and output is renderers, renderer extension will get loaded and output displayed.
• Renderer extension could send messages to extension host, however its possible the extension Extension A has not yet been activated, hence the message is never handled.

Solution 1

• Assume it takes a few seconds for Extension A to get activated, the renderer Extension could send messages every 500ms for the next 10s, until it gets a response from Extension A.
• Cons:
  • What's the frequency of these messages & how long should it keep sending these messages
  • Doesn't work well if there are multiple Extensions that need to communicate with Renderer

Solution 2

• When renderer A sends a message to the extension host, VS Code could store this in some internal buffer.
• When the extension A is up and running, VS Code could empty the buffer.
• Cons:
  • What happens when we have Extension B that also wants to listen to messages. Do we re-play these messages to extension B as well. This could be incorrect in some situations as Extension B might not be interested in stale communication messages.
  • Other issues arising from unnecessarily buffering messages in vscode (unnecessarily chewing up resource by buffering)
  • What if the other extensions never activate?

Solution 3

• When renderer Extension R is up and running, it will:
  • Broadcast (to extension host) the fact that it is up and running.
  • Listing to incoming messages from extension
• Similarly, Extension A will:
  • Broadcast (to renderer) the fact that it is up and running.
  • Listing to incoming messages from renderer extension
• This way, if we have another Extension B, it will be able to communicate with Renderer Extension R as well.
• Cons:
  • The Extensions will need to come up with their own messages thats used to commence the handshake.

[rebornix]

The solution 3 is always needed since currently extensions always need to create a messaging object https://github.com/microsoft/vscode/blob/main/src/vs/vscode.d.ts#L12356 to communicate with the renderers loaded in the notebook webview, since the extension code and the renderers are loaded in different processes, there is no guarantee which piece of code finish evaluation and execution first. In other words, if one wants to build a renderer which can talk to the extension part, a handshake is always required as there is no guarantee which code is ready first.

[rebornix]

Summary of how one renderer extension would need to do to support communication between the renderer code in webview and extension code running in the extension host code.

For example, a renderer extension contributes a renderer renderer_a for mimetype application/vnd-sample. The renderer extension code are activated separately in two contexts

1. Users open a notebook, which contain an output with mimetype application/vnd-sample.
2. Extension code gets activated by onNotebook event.
   2.1 Create a communication object by const comm = vscode.notebooks.createRendererMessaging('renderer_a'). and use comm.postMessage and comm.onDidReceiveMessage
3. Renderer code loaded in Notebook webview
   3.1 Users scroll the output into view
   3.2 Renderer code gets activated, and use context.postMessage and context.onDidReceiveMessage to communicate with the extension side

There is no guarantee when 2.1 or 3.2 is executed or in what sequence. We can see them as two processes and the code on each side talk to each other through { postMessage, onDidRecieveMessage }. Both sides should still work even if the other side is not ready yet or become unresponsive.

[sandy081]

@DonJayamanne May I know if this needs to be verified (if so add verification-needed label) or tested via test plan (if so add on-testplan label)?

[DonJayamanne]

@sandy081 Sorry, the documentation needs to be udpated. Will work on that today.