Add anti-affinity rules to controller's replicas

[jkhelil]

As part of the HA setup, we want to add anti-affinity rules to our deployment to make sure replicas do not end up in the same node.

Changes

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

• Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
• Has Tests included if any functionality added or changed
• pre-commit Passed
• Follows the commit message standard
• Meets the Tekton contributor standards (including functionality, content, code)
• Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
• Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
• Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

As part of improving high availability (HA) for the Tekton Pipelines controller, this update adds a preferredDuringSchedulingIgnoredDuringExecution pod anti-affinity rule. This ensures that multiple replicas of the controller are preferably scheduled on different nodes, reducing the risk of service disruption in case of a node failure.

[vdemeester]

/kind feature
/cc @afrittoli

[tekton-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[vdemeester]

@jkhelil we probably want to have a release-note entry for this one 👼🏼

[vdemeester]

/lgtm