Add Voila configuration option for default kernel environmental variables (#1175)

* Adds default kernel env varaibles to configuration options

* Adds test for default kernel env variables

* Docs update for default kernal env variables CLI option

* Move default_env_variables to ViolaKernelManager

* Update documentation

Co-authored-by: dcnadler <[email protected]>
Co-authored-by: Duc Trung LE <[email protected]>

Author: 3 people

docs/source/customize.rst

@@ -424,13 +424,14 @@ Preheated kernels
 ==================
 
 Since Voilà needs to start a new jupyter kernel and execute the requested notebook in this kernel for every connection, this would lead to a long waiting time before the widgets can be displayed in the browser. 
-To reduce this waiting time, especially for the heavy notebooks, users can activate the preheating kernel option of Voilà.
+To reduce this waiting time, especially for heavy notebooks, users can activate the preheating kernel option of Voilà.
 
 .. warning::
    Because preheated kernels are not executed on request, this feature is incompatible with the `prelaunch-hook` functionality.
 
 This option will enable two features:
 
+
 - A pool of kernels is started for each notebook and kept in standby, then the notebook is executed in every kernel of its pool. When a new client requests a kernel, the preheated kernel in this pool is used and another kernel is started asynchronously to refill the pool.
 - The HTML version of the notebook is rendered in each preheated kernel and stored, when a client connects to Voila, under some conditions, the cached HTML is served instead of re-rendering the notebook.
 
@@ -440,7 +441,15 @@ The preheating kernel option works with any kernel manager, it is deactivated by
 
     voila --preheat_kernel=True --pool_size=5
 
-If the pool size does not match the user's requirements, or some notebooks need to use environment variables..., additional settings are needed.  The easiest way to change these settings is to provide a file named `voila.json` in the same folder containing the notebooks. Settings for preheating kernel ( list of notebooks does not need preheated kernels, number of kernels in pool, refilling delay, environment variables for starting kernel...) can be set under the `VoilaKernelManager` class name.
+The default environment variables for preheated kernels can be set by the `VoilaKernelManager.default_env_variables` setting. For example, this command
+
+.. code-block:: bash
+
+    voila --preheat_kernel=True --VoilaKernelManager.default_env_variables='{"FOO": "BAR"}'
+
+will set the variable "FOO" in all preheated kernels.
+
+If the pool size does not match the user's requirements, or some notebooks need to use specific environment variables..., additional settings are needed.  The easiest way to change these settings is to provide a file named `voila.json` in the same folder containing the notebooks. Settings for preheating kernel ( list of notebooks does not need preheated kernels, number of kernels in pool, refilling delay, environment variables for starting kernel...) can be set under the `VoilaKernelManager` class name.
 
 Here is an example of settings with explanations for preheating kernel option. 
 
@@ -519,7 +528,7 @@ In preheating kernel mode, users can prepend with ``wait_for_request`` from ``vo
 
 ``wait_for_request`` will pause the execution of the notebook in the preheated kernel at this cell and wait for an actual user to connect to Voilà, set the request info environment variables and then continue the execution of the remaining cells.
 
-If the Voilà websocket handler is not started with the default protocol (`ws`), the default IP address (`127.0.0.1`) the default port (`8866`) or with url suffix, users need to provide these values through the environment variables ``VOILA_WS_PROTOCOL``, ``VOILA_APP_IP``, ``VOILA_APP_PORT`` and ``VOILA_WS_BASE_URL``. The easiest way is to set these variables in the `voila.json` configuration file, for example:
+If the Voilà websocket handler is not started with the default protocol (`ws`), the default IP address (`127.0.0.1`) the default port (`8866`) or with url suffix, users need to provide these values through the environment variables ``VOILA_WS_PROTOCOL``, ``VOILA_APP_IP``, ``VOILA_APP_PORT`` and ``VOILA_WS_BASE_URL``. One way to set these variables is in the `voila.json` configuration file, for example:
 
 .. code-block:: python
 
@@ -540,6 +549,12 @@ If the Voilà websocket handler is not started with the default protocol (`ws`),
       }
    }
 
+Additionally, you can set these with the command:
+
+.. code-block:: bash
+
+    voila --preheat_kernel=True --VoilaKernelManager.default_env_variables='{"VOILA_WS_PROTOCOL":"wss","VOILA_APP_IP":"192.168.1.1"}'
+
 Hiding output and code cells based on cell tags
 ===============================================
 

tests/app/preheat_default_kernel_env_test.py

@@ -0,0 +1,35 @@
+import asyncio
+import os
+
+import pytest
+
+
+@pytest.fixture()
+def voila_args_extra():
+    return ['--VoilaKernelManager.default_env_variables={"FOO": "BAR"}']
+
+
+@pytest.fixture
+def preheat_mode():
+    return True
+
+
+@pytest.fixture
+def voila_notebook(notebook_directory):
+    return os.path.join(notebook_directory, 'preheat', 'default_env_variables.ipynb')
+
+
+NOTEBOOK_EXECUTION_TIME = 2
+
+
+async def send_request(sc, url, wait=0):
+    await asyncio.sleep(wait)
+    response = await sc.fetch(url)
+    return response.body.decode('utf-8')
+
+
+async def test_default_kernel_env_variable(http_server_client, base_url):
+    html_text = await send_request(sc=http_server_client,
+                                   url=base_url,
+                                   wait=NOTEBOOK_EXECUTION_TIME + 1)
+    assert 'BAR' in html_text

tests/notebooks/preheat/default_env_variables.ipynb

@@ -0,0 +1,36 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7efe3e8f-4b8d-4fd3-99d1-b06d79b88ae2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import time\n",
+    "time.sleep(2)\n",
+    "print('hello world')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "782377cd-2bc0-4d0d-8b63-74dcb7e9d645",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "os.getenv('FOO')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3.10.5 ('voila')",
+   "language": "python",
+   "name": "python3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

voila/app.py

@@ -457,7 +457,7 @@ def start(self):
         kernel_manager_class = voila_kernel_manager_factory(
             self.voila_configuration.multi_kernel_manager_class,
             preheat_kernel,
-            pool_size
+            pool_size,
         )
         self.kernel_manager = kernel_manager_class(
             parent=self,

voila/voila_kernel_manager.py

@@ -41,6 +41,8 @@ def voila_kernel_manager_factory(base_class: Type[T], preheat_kernel: bool, defa
     Args:
         - base_class (Type[T]): The kernel manager class
         - preheat_kernel (Bool): Flag to decorate the input class
+        - default_pool_size (int): Size of pre-heated kernel pool for each notebook.
+            Zero or negative number means disabled
 
     Returns:
         T: Decorated class
@@ -73,6 +75,13 @@ class VoilaKernelManager(base_class):
                 variables used to start kernel''',
             )
 
+            default_env_variables = Dict(
+                {},
+                config=True,
+                help='''Default environmental variables for kernels
+                ''',
+            )
+
             preheat_blacklist = List([],
                                      config=True,
                                      help='List of notebooks which do not use pre-heated kernel.')
@@ -85,7 +94,12 @@ class VoilaKernelManager(base_class):
 
             @default('kernel_pools_config')
             def _kernel_pools_config(self):
-                return {'default': {'pool_size': max(default_pool_size, 0), 'kernel_env_variables': {}}}
+                return {
+                    'default': {
+                        'pool_size': max(default_pool_size, 0),
+                        'kernel_env_variables': self.default_env_variables,
+                    }
+                }
 
             def __init__(self, **kwargs):