Istio sidecar injection through annotation doesn't work

[kromanow94]

Validation Checklist

• Is this a Kubeflow issue?
• Are you posting in the right repository ?
• Did you follow the installation guide https://github.com/kubeflow/manifests?tab=readme-ov-file ?
• Is the issue report properly structured and detailed with version numbers?
• Is this for Kubeflow development ?
• Would you like to work on this issue?
• You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

Version

master

Describe your issue

In the Istio docs Installing the Sidecar#Controlling the injection policy, the described way of controlling the istio sidecar injection is through the labels, both for the Pods and Namespaces. In the manifests, a lot of resources seemingly tries to control the sidecar injection behavior through annotation, which just doesn't work.

We should investigate which pods really need the sidecars and comply with the proper istio injection mechanism through labels.

Steps to reproduce the issue

$ kustomize build example/ | yq 'select(.spec.template.metadata.annotations."sidecar.istio.io/inject") | {kind: .kind, name: .metadata.name, istioInjectionValue: .spec.template.metadata.annotations."sidecar.istio.io/inject"}' -y

kind: Deployment
name: cluster-local-gateway
istioInjectionValue: 'false'
---
kind: Deployment
name: istio-ingressgateway
istioInjectionValue: 'false'
---
kind: Deployment
name: istiod
istioInjectionValue: 'false'
---
kind: Deployment
name: activator
istioInjectionValue: 'true'
---
kind: Deployment
name: autoscaler
istioInjectionValue: 'true'
---
kind: Deployment
name: controller
istioInjectionValue: 'true'
---
kind: Deployment
name: net-istio-controller
istioInjectionValue: 'true'
---
kind: Deployment
name: net-istio-webhook
istioInjectionValue: 'true'
---
kind: Deployment
name: webhook
istioInjectionValue: 'true'
---
kind: Deployment
name: admission-webhook-deployment
istioInjectionValue: 'false'
---
kind: Deployment
name: centraldashboard
istioInjectionValue: 'true'
---
kind: Deployment
name: katib-controller
istioInjectionValue: 'false'
---
kind: Deployment
name: katib-db-manager
istioInjectionValue: 'false'
---
kind: Deployment
name: katib-mysql
istioInjectionValue: 'false'
---
kind: Deployment
name: katib-ui
istioInjectionValue: 'true'
---
kind: Deployment
name: kserve-controller-manager
istioInjectionValue: 'false'
---
kind: Deployment
name: kserve-models-web-app
istioInjectionValue: 'true'
---
kind: Deployment
name: kubeflow-pipelines-profile-controller
istioInjectionValue: 'false'
---
kind: Deployment
name: metadata-envoy-deployment
istioInjectionValue: 'false'
---
kind: Deployment
name: profiles-deployment
istioInjectionValue: 'true'
---
kind: Deployment
name: training-operator
istioInjectionValue: 'false'
---
kind: StatefulSet
name: metacontroller
istioInjectionValue: 'false'

Put here any screenshots or videos (optional)

No response

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[kromanow94]

Not stale.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[juliusvonkohout]

/lifecycle frozen