Allow configuration of waitForPod timeout

[luqic]

Hello!

Is it possible to allow the configuration of the waitForPod timeout?

thanos-receive-controller/main.go

Line 605 in c57219e

return wait.PollImmediate(time.Second, time.Minute, func() (bool, error) {

It's currently hardcoded as 1 minute, but pods in my cluster can take longer than that to start up (e.g. when spinning up a new node), resulting in this warning: level=warn caller=main.go:573 ts=2024-01-11T09:24:33.383554728Z msg="failed polling until pod is ready" pod=thanos-receive-3 duration=1m0.010513876s err="timed out waiting for the condition". This results in the Hashring ConfigMap getting updated before all pods are ready.

FYI we have --allow-only-ready-replicas enabled and are on image version main-2023-11-06-c57219e.