Add graceful termination to coturn chart (#2456)

* charts/coturn: add support for graceful termination.

This adds a preStop hook to coturn pods which waits until the number of
active allocations goes to zero before the coturn process is signalled to
terminate. This can be used with an external service discovery mechanism to
enable graceful rolling restarts, by removing each running coturn pod from the
set of active advertised coturn instances in turn.

* changelog: add entry for coturn graceful termination feature.

* charts/coturn: update coturn container image version.

Starting with this version, our coturn image ships with the pre-stop hook
script, which is required for graceful termination.

Author: sysvinit

changelog.d/2-features/coturn-graceful-termination

@@ -0,0 +1,5 @@
+The coturn chart has new functionality to enable graceful pod termination, by
+waiting for all active allocations on a coturn instance to drain first. When
+combined with a suitable external service discovery mechanism which can steer
+client traffic away from terminating coturn pods, this can be used to implement
+graceful rolling restarts of clusters of coturn instances.

charts/coturn/Chart.yaml

@@ -11,4 +11,4 @@ version: 0.0.42
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
 # follow Semantic Versioning. They should reflect the version the application is using.
-appVersion: 4.5.2-wireapp.1
+appVersion: 4.5.2-wireapp.5

charts/coturn/templates/statefulset.yaml

@@ -93,6 +93,15 @@ spec:
               ZREST_SECRET="$(cat /secrets/zrest_secret.txt)"
               sed -Ee "s;__COTURN_EXT_IP__;$EXTERNAL_IP;g" -e "s;__COTURN_POD_IP__;$POD_IP;g" -e "s;__COTURN_SECRET__;$ZREST_SECRET;" /coturn-template/coturn.conf.template > /coturn-config/turnserver.conf
               exec /usr/bin/turnserver -c /coturn-config/turnserver.conf
+          {{- if .Values.coturnGracefulTermination }}
+          lifecycle:
+            preStop:
+              exec:
+                command:
+                  - /bin/sh
+                  - -c
+                  - exec /usr/local/bin/pre-stop-hook "$POD_IP" {{ .Values.coturnMetricsListenPort }}
+          {{- end }}
 
           ports:
             - name: coturn-tcp
@@ -114,6 +123,9 @@ spec:
 
           resources:
             {{- toYaml .Values.resources | nindent 12 }}
+      {{- if .Values.coturnGracefulTermination }}
+      terminationGracePeriodSeconds: {{ .Values.coturnGracePeriodSeconds }}
+      {{- end }}
       {{- with .Values.nodeSelector }}
       nodeSelector:
         {{- toYaml . | nindent 8 }}

charts/coturn/values.yaml

@@ -24,3 +24,15 @@ securityContext:
 
 coturnTurnListenPort: 3478
 coturnMetricsListenPort: 9641
+
+# This chart optionally supports waiting for traffic to drain from coturn
+# before pods are terminated. Warning: coturn does not have any way to steer
+# incoming client traffic away from itself on its own, so this functionality
+# relies on external traffic management (e.g. service discovery for active coturn
+# instances) to prevent clients from sending new requests to pods which are in a
+# terminating state.
+coturnGracefulTermination: false
+# Grace period for terminating coturn pods, after which they will be forcibly
+# terminated. This setting is only effective when coturnGracefulTermination is
+# set to true.
+coturnGracePeriodSeconds: 86400 # one day