Restart Property in ClickHouseInstallation spec not cleared after restart is completed

[evangeline013]

Environment:
Operator version: altinity/clickhouse-operator:0.24.5
ClickHouse version: 23.12.5.81

Issue:
When performing a rolling restart of ClickHouse via the operator by patching the restart property in the ClickHouseInstallation spec with the following command:

kubectl patch chi ch -n clickhouse --type=merge -p '{"spec":{"restart":"RollingUpdate"}}'

the restart works correctly the first time. However, the restart property remains in the spec after the restart completes, even though the documentation mentioned that the property is ephemeral and should be removed by the operator once the restart is complete.

Due to the Restart property not being removed, the same command will not trigger a restart since there's no change:

clickhouseinstallation.clickhouse.altinity.com/ch patched (no change)

I posted this in the slack help channel and Robert suggested that this behavior is not expected, so logging the issue here wondering if you guys can take a look

Also, would it be a safe workaround if we manually reset the field to an empty string before re-running the patch command like this?

kubectl patch chi ch -n clickhouse --type=merge -p '{"spec":{"restart":""}}'
kubectl patch chi ch -n clickhouse --type=merge -p '{"spec":{"restart":"RollingUpdate"}}'

[Slach]

if you just need trigger reconcile you an try
kubectl patch chi ch -n clickhouse --type=merge -p '{"spec":{"taskID":"any-random-string"}}'

[evangeline013]

Thanks for the quick response! Some questions before we adopt the taskID approach:

1. Is spec.taskID considered an internal/ephemeral field purely to trigger a reconcile, and is it safe to bump at any time?

2. Some context for our use case, we are using the cert-manager to manage our TLS Certificates, and we have seen that after the certs got renewed, the updated files are correctly mounted into the pods, but ClickHouse and Keeper did not reload the new certs and as a result we’re seeing SSL handshake errors until we manually deleted the pods, which then forcing new pods to be created and the servers restarted to pick up the new certificates. Would the command to trigger reconcile help us achieve the same goal?

3. Does the operator ever overwrite/clear taskID, or should we expect it to persist until we change it again?

4. Is the taskID reconcile trigger also supported for ClickHouseKeeperInstallation (CHK) resources?

[hodgesrm]

Follow-up on your questions.

1. @Slach 's suggestion is the correct way to trigger reconciliation. However, it won't necessarily lead to a restart.
2. It turns out the restart field is not cleared by design. The operator does not alter CHI contents. That also means that taskID values stick around as well. They won't change unless you change them.
3. You can bump taskID at any time. It's only for reconciliation.
4. Restart happens under a variety of conditions. You can try to bump taskID to see if it will cause a restart, but it is not guaranteed except in obvious cases like changes that rewrite the pod spec.
5. I checked and the taskID is also present in the CHK resource, so it should work just as it does in CHI. You can confirm by setting it as a property of the spec: section. You should see it trigger reconciliation in the operator logs.

[Slach]

ClickHouse and Keeper didnot reload the new certs and as a result we’re seeing SSL handshake errors

Which clickhouse-server and clickhouse-keeper versions do you use?
did you try on 25.3+?