-
Notifications
You must be signed in to change notification settings - Fork 10.3k
Description
Name and Version
bitnami/redis 20.0.5
What architecture are you using?
amd64
What steps will reproduce the bug?
- run chart with useHostnames=false (due to DNS issues causing tilt mode)
- enable sentinel
- enable masterService
Are you using any custom parameters or values?
useHostnames: false
sentinel:
enabled: true
masterService:
enabled: true
What is the expected behavior?
master pod is labeled with isMaster="true"
What do you see instead?
no such label on master
Additional information
kubectl uses the content of /etc/shared/current
for the pod selector in update-master-label.sh
.
The content of /etc/shared/current
is written by sentinel from ${REDIS_MASTER_HOST/.*}
in start-sentinel.sh
.
when using useHostnames=false REDIS_MASTER_HOST contains not redis-node-0.redis.svc.cluster.local
, which would return redis-node-0
as current
, but instead the IP address e.g. 172.22.140.140
which returns 172
as current
.
thus kubectl matches no pod and updates no labels.
Unfortunately, the output of kubectl is also not logged - only the echo statement with new master elected, updating label(s)...
.
Patching the script configmap with echo "$(< "/etc/shared/current")"
helped to debug this as it returned 172
instead of redis-node-0
.
This also applies to the delete patch for the label app.kubernetes.io/role
which will be fixed in 20.0.6 (annotation -> label).
To fix this either sentinel writes a non-resolved value for REDIS_MASTER_HOST
into /etc/shared/current
or the kubectl script looks up the pod by IP if current
is numerical. such as:
kubectl get -n <namespace> --output json pods | jq '.items[] | select(.status.podIP=="<IP>")' | jq .metadata.name
though this would likely require further RBAC permissions.