services: add label app to services to be able to use a labelSelector #7371

rphillips · 2025-10-24T00:35:39Z

What type of PR is this?

/kind bug

What this PR does / why we need it:

This PR is trivial though important. There is not a current way to individually select a service by a labelSelector. For instance, control-plane: controller-manager matches numerous components.

It becomes more important when wiring up Prometheus to a specific endpoint. Not all the services export a /metrics endpoint thus causing Prometheus to fail on requests to that endpoint, so matching on everything with control-plane: controller-manager does not work.

The PR adds a unique label - the same name as the service - so a labelSelector can be used to select a particular service.

I would like to request this patch be backported to 0.14.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Services: fix the setting of the `app.kubernetes.io/component` label to discriminate between different service components within Kueue as follows:
- controller-manager-metrics-service for kueue-controller-manager-metrics-service 
- visibility-service for kueue-visibility-server
- webhook-service for kueue-webhook-service

netlify · 2025-10-24T00:35:45Z

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name	Link
🔨 Latest commit	`d12fd59`
🔍 Latest deploy log	https://app.netlify.com/projects/kubernetes-sigs-kueue/deploys/69035ac719b26e00080937fc

k8s-ci-robot · 2025-10-24T00:35:49Z

Hi @rphillips. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

kannon92 · 2025-10-24T00:44:50Z

/ok-to-test

mimowo · 2025-10-24T07:22:18Z

@rphillips have you had a chance to test end-to-end manually if this works well on upgrade?

Ideally we test it both for manifests-based version and Helm

rphillips · 2025-10-24T16:37:44Z

@mimowo I had claude help me write two tests (which did find a bug, which I fixed). The gist has two test files and can be ran in the kueue repo. They install kueue 0.14.2 using kind, then upgrade to the local helm or kustomize charts. Then assert the services have the 'app' label.

Both tests pass.

mimowo · 2025-10-27T14:11:53Z

This PR is trivial though important. There is not a current way to individually select a service by a labelSelector. For instance, control-plane: controller-manager matches numerous components.

I agree being able to select services of a specific application like Kueue is important, but I we already have the app.kubernetes.io/name label for this purpose. I tested locally installing Kueue both from manifest and Helm and got results.

after Helm installation:

❯ k get svc -nkueue-system --show-labels
NAME                                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE   LABELS
kueue-controller-manager-metrics-service   ClusterIP   10.96.227.177   <none>        8443/TCP   35m   app.kubernetes.io/component=metrics,app.kubernetes.io/instance=kueue,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=kueue,app.kubernetes.io/version=v0.14.2,control-plane=controller-manager,helm.sh/chart=kueue-0.14.2
kueue-visibility-server                    ClusterIP   10.96.7.227     <none>        443/TCP    35m   app.kubernetes.io/instance=kueue,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=kueue,app.kubernetes.io/version=v0.14.2,control-plane=controller-manager,helm.sh/chart=kueue-0.14.2
kueue-webhook-service                      ClusterIP   10.96.114.17    <none>        443/TCP    35m   app.kubernetes.io/instance=kueue,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=kueue,app.kubernetes.io/version=v0.14.2,control-plane=controller-manager,helm.sh/chart=kueue-0.14.2

after manifest installation:

❯ k get svc -nkueue-system --show-labels
NAME                                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE   LABELS
kueue-controller-manager-metrics-service   ClusterIP   10.96.128.161   <none>        8443/TCP   44s   app.kubernetes.io/component=controller,app.kubernetes.io/managed-by=kustomize,app.kubernetes.io/name=kueue,control-plane=controller-manager
kueue-visibility-server                    ClusterIP   10.96.204.159   <none>        443/TCP    44s   app.kubernetes.io/component=controller,app.kubernetes.io/name=kueue,control-plane=controller-manager
kueue-webhook-service                      ClusterIP   10.96.212.149   <none>        443/TCP    44s   app.kubernetes.io/component=controller,app.kubernetes.io/name=kueue,control-plane=controller-manager

I think what could be improved clearly is the use of the app.kubernetes.io/component label to say specificly which component of Kueue this is.

Also, instead of the generic "app" label I prefer to make sure the standard app.kubernetes.io/name is used well.

Let me know if I'm missing something, and let me hold until this is clarfied.
/hold

rphillips · 2025-10-27T15:42:08Z

No problem.

I updated the helm chart to result in:

NAME                                       LABELS
kueue-controller-manager-metrics-service   map[app.kubernetes.io/component:controller-manager-metrics-service app.kubernetes.io/instance:kueue app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:kueue app.kubernetes.io/version:v0.14.2 control-plane:controller-manager helm.sh/chart:kueue-0.14.2]
kueue-visibility-server                    map[app.kubernetes.io/component:visibility-service app.kubernetes.io/instance:kueue app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:kueue app.kubernetes.io/version:v0.14.2 control-plane:controller-manager helm.sh/chart:kueue-0.14.2]
kueue-webhook-service                      map[app.kubernetes.io/component:webhook-service app.kubernetes.io/instance:kueue app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:kueue app.kubernetes.io/version:v0.14.2 control-plane:controller-manager helm.sh/chart:kueue-0.14.2]

And the kustomize charts to:

NAME                                       LABELS
kueue-controller-manager-metrics-service   map[app.kubernetes.io/component:controller-manager-metrics-service app.kubernetes.io/managed-by:kustomize app.kubernetes.io/name:kueue control-plane:controller-manager]
kueue-visibility-server                    map[app.kubernetes.io/component:visibility-service app.kubernetes.io/name:kueue control-plane:controller-manager]
kueue-webhook-service                      map[app.kubernetes.io/component:webhook-service app.kubernetes.io/name:kueue control-plane:controller-manager]

Are there any other changes you would like to see?

mimowo · 2025-10-27T15:52:11Z

The output for Helm looks great. I just need another test if the upgrade works well, unfortunately we don't have automated upgrade tests and this change has a potential impact.

The output for installation from manifests contains app.kubernetes.io/managed-by:Helm so something looks off.

rphillips · 2025-10-27T16:31:22Z

Yep. Sorry. Copy and paste error on the kustomize output (fixed). I also updated the gist with the latest test scripts I used. Also fixed pull-kueue-verify-main.

mimowo · 2025-10-29T19:06:02Z

@rphillips I think as we should update the release note as the PR changed.

Proposal:

/release-note-edit

Services: fix the setting of the `app.kubernetes.io/component` label to discriminate between different service components within Kueue as follows:
- controller-manager-metrics-service for kueue-controller-manager-metrics-service 
- visibility-service for kueue-visibility-server
- webhook-service for kueue-webhook-service

wdyt?

rphillips · 2025-10-29T19:12:28Z

Works for me! Looks like the PR got updated.

mimowo · 2025-10-29T19:20:01Z

Thank you 👍
/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

k8s-infra-cherrypick-robot · 2025-10-29T19:20:03Z

@mimowo: once the present PR merges, I will cherry-pick it on top of release-0.13, release-0.14 in new PRs and assign them to you.

In response to this:

Thank you 👍
/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2025-10-29T19:20:08Z

LGTM label has been added.

Git tree hash: 35a11dd42e454ea9f895d2a39ff2b553b032b7aa

tenzen-y · 2025-10-29T19:28:09Z

config/default/kustomization.yaml

 labels:
  - pairs:
      app.kubernetes.io/name: kueue
-      app.kubernetes.io/component: controller


Doesn't this drop the app.kubernetes.io/component label from the kueue-controller-manager deployment?

We have to fix this slightly different. This specific spot adds app.kubernetes.io/component=controller to all the other components. I'll update the PR.

I pushed the fix.

mimowo · 2025-10-29T20:03:36Z

/hold
to verify #7371 (comment)

rphillips · 2025-10-29T20:20:26Z

I don't see app.kubernetes.io/component set with 0.14.2 on controller-manager with Helm, but I'll fix that.

With this PR and helm:

kueue-controller-manager   1/1     1            1           38s   app.kubernetes.io/component=controller,app.kubernetes.io/instance=kueue,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=kueue,app.kubernetes.io/version=v0.14.2,control-plane=controller-manager,helm.sh/chart=kueue-0.14.2

NAME                                       LABELS
kueue-controller-manager-metrics-service   map[app.kubernetes.io/component:controller-manager-metrics-service app.kubernetes.io/instance:kueue app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:kueue app.kubernetes.io/version:v0.14.2 control-plane:controller-manager helm.sh/chart:kueue-0.14.2]
kueue-visibility-server                    map[app.kubernetes.io/component:visibility-service app.kubernetes.io/instance:kueue app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:kueue app.kubernetes.io/version:v0.14.2 control-plane:controller-manager helm.sh/chart:kueue-0.14.2]
kueue-webhook-service                      map[app.kubernetes.io/component:webhook-service app.kubernetes.io/instance:kueue app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:kueue app.kubernetes.io/version:v0.14.2 control-plane:controller-manager helm.sh/chart:kueue-0.14.2]

With this PR and kustomize:

kueue-controller-manager   1/1     1            1           34s   app.kubernetes.io/component=controller,app.kubernetes.io/name=kueue,control-plane=controller-manager

NAME                                       LABELS
kueue-controller-manager-metrics-service   map[app.kubernetes.io/component:controller-manager-metrics-service app.kubernetes.io/managed-by:kustomize app.kubernetes.io/name:kueue control-plane:controller-manager]
kueue-visibility-server                    map[app.kubernetes.io/component:visibility-service app.kubernetes.io/name:kueue control-plane:controller-manager]
kueue-webhook-service                      map[app.kubernetes.io/component:webhook-service app.kubernetes.io/name:kueue control-plane:controller-manager]

mimowo · 2025-10-30T08:04:56Z

config/default/kustomization.yaml

+      name: controller-manager-metrics-service
+      namespace: system
+      labels:
+        app.kubernetes.io/component: controller-manager-metrics-service


Suggested change

app.kubernetes.io/component: controller-manager-metrics-service

app.kubernetes.io/component: metrics-service

I want to propose simpler name here. We don't need to rename the service name in this PR, just the label. The name of the labels does not need to match the service name exactly anyway as the service names have the "kueue-" prefix anyway. wdyt? cc @tenzen-y

I think that we should have a prefix since users typically deploy the external manager including job controller implementing jobframework and external dispatchers to the same namespace.

by prefix do you mean controller-manager- or kueue-?

In any case I don't think this is needed, because we have another standard app.kubernetes.io/name label which sets the context. So users who write label selectors can use both app.kubernetes.io/name: kueue and app.kubernetes.io/component: metrics-service.

I think this is the intention of the app.kubernetes.io/component is not to repeat the app name, see examples here: https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels

# This is an excerpt apiVersion: apps/v1 kind: StatefulSet metadata: labels: app.kubernetes.io/name: mysql app.kubernetes.io/instance: mysql-abcxyz app.kubernetes.io/version: "5.7.21" app.kubernetes.io/component: database app.kubernetes.io/part-of: wordpress app.kubernetes.io/managed-by: Helm

That makes sense.
In that case, the service for keueu external-controller-manager could have app.kubernetes.io/name: kueue-external-manager and app.kubernetes.io/instance: metrics-service label.

what is kueue external-controller-manager ? TBH I think the app name is simply "kueue".

Kueue Controller Manager is hosted in this repository.
Kueue External Controller Manager is served in users's respotisoty which has custom JobFramework controller and external dispatchers.

In this repository, we use just app.kubernetes.io/instance: metrics-service and app.kubernetes.io/name: kueue.

I see, thank you for clarifying 👍

I'll commit this and squash the commits. Thank you for the feedback.

rphillips · 2025-10-30T12:31:32Z

Updated.

With this PR and helm:

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE   LABELS
kueue-controller-manager   1/1     1            1           38s   app.kubernetes.io/component=controller,app.kubernetes.io/instance=kueue,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=kueue,app.kubernetes.io/version=v0.14.2,control-plane=controller-manager,helm.sh/chart=kueue-0.14.2

NAME                                       LABELS
kueue-controller-manager-metrics-service   map[app.kubernetes.io/component:metrics-service app.kubernetes.io/instance:kueue app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:kueue app.kubernetes.io/version:v0.14.2 control-plane:controller-manager helm.sh/chart:kueue-0.14.2]
kueue-visibility-server                    map[app.kubernetes.io/component:visibility-service app.kubernetes.io/instance:kueue app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:kueue app.kubernetes.io/version:v0.14.2 control-plane:controller-manager helm.sh/chart:kueue-0.14.2]
kueue-webhook-service                      map[app.kubernetes.io/component:webhook-service app.kubernetes.io/instance:kueue app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:kueue app.kubernetes.io/version:v0.14.2 control-plane:controller-manager helm.sh/chart:kueue-0.14.2]

With this PR and kustomize:

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE   LABELS
kueue-controller-manager   1/1     1            1           34s   app.kubernetes.io/component=controller,app.kubernetes.io/name=kueue,control-plane=controller-manager

NAME                                       LABELS
kueue-controller-manager-metrics-service   map[app.kubernetes.io/component:metrics-service app.kubernetes.io/managed-by:kustomize app.kubernetes.io/name:kueue control-plane:controller-manager]
kueue-visibility-server                    map[app.kubernetes.io/component:visibility-service app.kubernetes.io/name:kueue control-plane:controller-manager]
kueue-webhook-service                      map[app.kubernetes.io/component:webhook-service app.kubernetes.io/name:kueue control-plane:controller-manager]

mimowo · 2025-10-30T12:59:53Z

/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

k8s-infra-cherrypick-robot · 2025-10-30T12:59:57Z

@mimowo: once the present PR merges, I will cherry-pick it on top of release-0.13, release-0.14 in new PRs and assign them to you.

In response to this:

/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2025-10-30T13:00:02Z

LGTM label has been added.

Git tree hash: 2d7ac314faa9078832209e45c068b7704cb0f8ce

k8s-ci-robot · 2025-10-30T13:00:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mimowo, rphillips

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [mimowo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

rphillips · 2025-10-30T13:11:12Z

/hold cancel

k8s-infra-cherrypick-robot · 2025-10-30T13:38:44Z

@mimowo: new pull request created: #7450

In response to this:

/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-infra-cherrypick-robot · 2025-10-30T13:39:21Z

@mimowo: new pull request created: #7451

In response to this:

/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. labels Oct 24, 2025

k8s-ci-robot requested a review from kannon92 October 24, 2025 00:35

k8s-ci-robot requested a review from PBundyra October 24, 2025 00:35

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 24, 2025

k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Oct 24, 2025

rphillips mentioned this pull request Oct 24, 2025

WIP: fix metrics openshift/kueue-operator#856

Open

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 24, 2025

rphillips mentioned this pull request Oct 24, 2025

fix kueue service monitor with correct port to scrape metrics openshift/kueue-operator#852

Open

rphillips force-pushed the add_label_app_to_services branch from e86f8d7 to bf879bb Compare October 24, 2025 01:21

rphillips force-pushed the add_label_app_to_services branch from bf879bb to 8a9457e Compare October 24, 2025 16:32

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Oct 24, 2025

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 27, 2025

rphillips force-pushed the add_label_app_to_services branch from 8a9457e to a5c43d2 Compare October 27, 2025 15:42

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 27, 2025

rphillips force-pushed the add_label_app_to_services branch from a5c43d2 to 9743ab4 Compare October 27, 2025 16:30

k8s-ci-robot assigned mimowo Oct 29, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 29, 2025

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 29, 2025

tenzen-y reviewed Oct 29, 2025

View reviewed changes

rphillips force-pushed the add_label_app_to_services branch from 9743ab4 to e8542a4 Compare October 29, 2025 20:20

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 29, 2025

k8s-ci-robot requested a review from mimowo October 29, 2025 20:20

mimowo reviewed Oct 30, 2025

View reviewed changes

services: update app.kuberntes.io/component for services

d12fd59

rphillips force-pushed the add_label_app_to_services branch from e8542a4 to d12fd59 Compare October 30, 2025 12:32

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 30, 2025

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 30, 2025

k8s-ci-robot merged commit 0290088 into kubernetes-sigs:main Oct 30, 2025
23 checks passed

k8s-ci-robot added this to the v0.15 milestone Oct 30, 2025

k8s-infra-cherrypick-robot mentioned this pull request Oct 30, 2025

[release-0.14] services: add label app to services to be able to use a labelSelector #7450

Merged

k8s-infra-cherrypick-robot mentioned this pull request Oct 30, 2025

[release-0.13] services: add label app to services to be able to use a labelSelector #7451

Merged

	app.kubernetes.io/component: controller-manager-metrics-service
	app.kubernetes.io/component: metrics-service

services: add label app to services to be able to use a labelSelector #7371

services: add label app to services to be able to use a labelSelector #7371

Uh oh!

Conversation

rphillips commented Oct 24, 2025 • edited by k8s-ci-robot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Uh oh!

netlify bot commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Uh oh!

k8s-ci-robot commented Oct 24, 2025

Uh oh!

kannon92 commented Oct 24, 2025

Uh oh!

mimowo commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rphillips commented Oct 24, 2025

Uh oh!

mimowo commented Oct 27, 2025

Uh oh!

rphillips commented Oct 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mimowo commented Oct 27, 2025

Uh oh!

rphillips commented Oct 27, 2025

Uh oh!

mimowo commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rphillips commented Oct 29, 2025

Uh oh!

mimowo commented Oct 29, 2025

Uh oh!

k8s-infra-cherrypick-robot commented Oct 29, 2025

Uh oh!

k8s-ci-robot commented Oct 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mimowo commented Oct 29, 2025

Uh oh!

rphillips commented Oct 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mimowo Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rphillips commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mimowo commented Oct 30, 2025

Uh oh!

rphillips commented Oct 24, 2025 •

edited by k8s-ci-robot

Loading

netlify bot commented Oct 24, 2025 •

edited

Loading

mimowo commented Oct 24, 2025 •

edited

Loading

rphillips commented Oct 27, 2025 •

edited

Loading

mimowo commented Oct 29, 2025 •

edited

Loading

mimowo Oct 30, 2025 •

edited

Loading

rphillips commented Oct 30, 2025 •

edited

Loading