Prometheus port missing on a katib-controller pod

[alexeykaplin]

/kind bug

k8s resource inconsistency: service has port annotation, but pod has no port annotation.

Environment:
kubeflow 1.4.0
Default deployment.

• Kubeflow version (kfctl version): 1.4.1
• Kubernetes version: (use kubectl version): 1.19.8
• OS (e.g. from /etc/os-release): Linux