Skip to content

Commit c6ff981

Browse files
austinzhao-gotekton-robot
austinzhao-go
authored and
tekton-robot
committed
Support Task-level Resources Requirements on TaskRun: Part #1
Required fields and related webhook validations are added to support a user to configure compute resources for TaskRun which will significantly reduce the over-asked resources amount configured by the Step-level.
1 parent ea87e41 commit c6ff981

13 files changed

+344
-7
lines changed

‎docs/compute-resources.md‎

Lines changed: 84 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,10 @@ request will be 6.
4242
Since the sidecar container has no CPU limit, this is treated as the highest CPU limit.
4343
Therefore, the pod will have no effective CPU limit.
4444

45-
## Task Resource Requirements
45+
## Task-level Compute Resources Configuration
46+
47+
**([alpha only](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#alpha-features))**
48+
(This feature is under development and not functional yet. Stay tuned!)
4649

4750
Tekton allows users to specify resource requirements of [`Steps`](./tasks.md#defining-steps),
4851
which run sequentially. However, the pod's effective resource requirements are still the
@@ -52,6 +55,86 @@ requirements for `Step` containers, they must be treated as if they are running
5255
Tekton adjusts `Step` resource requirements to comply with [LimitRanges](#limitrange-support).
5356
[ResourceQuotas](#resourcequota-support) are not currently supported.
5457

58+
Instead of specifying resource requirements on each `Step`, users can choose to specify resource requirements at the Task-level. If users specify a Task-level resource request, it will ensure that the kubelet reserves only that amount of resources to execute the `Task`'s `Steps`.
59+
If users specify a Task-level resource limit, no `Step` may use more than that amount of resources.
60+
61+
Each of these details is explained in more depth below.
62+
63+
Some points to note:
64+
65+
- Task-level resource requests and limits do not apply to sidecars which can be configured separately.
66+
- Users may not configure the Task-level and Step-level resource requirements (requests/limits) simultaneously.
67+
68+
### Configure Task-level Compute Resources
69+
70+
Task-level resource requirements can be configured in `TaskRun.ComputeResources`, or `PipelineRun.TaskRunSpecs.ComputeResources`.
71+
72+
e.g.
73+
74+
```yaml
75+
apiVersion: tekton.dev/v1beta1
76+
kind: TaskRun
77+
metadata:
78+
name: foo
79+
spec:
80+
computeResources:
81+
requests:
82+
cpu: 1
83+
limits:
84+
cpu: 2
85+
```
86+
87+
The following TaskRun will be rejected, because it configures both step-level and task-level compute resource requirements:
88+
89+
```yaml
90+
kind: TaskRun
91+
spec:
92+
stepOverrides:
93+
- name: foo
94+
resources:
95+
requests:
96+
cpu: 1
97+
computeResources:
98+
requests:
99+
cpu: 2
100+
```
101+
102+
```yaml
103+
kind: PipelineRun
104+
spec:
105+
taskRunSpecs:
106+
- pipelineTaskName: foo
107+
stepOverrides:
108+
- name: foo
109+
resources:
110+
requests:
111+
cpu: 1
112+
computeResources:
113+
requests:
114+
cpu: 2
115+
```
116+
117+
### Configure Resource Requirements with Sidecar
118+
119+
Users can specify compute resources separately for a sidecar while configuring task-level resource requirements on TaskRun.
120+
121+
e.g.
122+
123+
```yaml
124+
kind: TaskRun
125+
spec:
126+
sidecarOverrides:
127+
- name: sidecar
128+
resources:
129+
requests:
130+
cpu: 750m
131+
limits:
132+
cpu: 1
133+
computeResources:
134+
requests:
135+
cpu: 2
136+
```
137+
55138
## LimitRange Support
56139
57140
Kubernetes allows users to configure [LimitRanges]((https://kubernetes.io/docs/concepts/policy/limit-range/)),

‎docs/install.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -426,6 +426,7 @@ Features currently in "alpha" are:
426426
| [Step and Sidecar Overrides](./taskruns.md#overriding-task-steps-and-sidecars) | [TEP-0094](https://github.com/tektoncd/community/blob/main/teps/0094-specifying-resource-requirements-at-runtime.md) | | |
427427
| [Matrix](./matrix.md) | [TEP-0090](https://github.com/tektoncd/community/blob/main/teps/0090-matrix.md) | | |
428428
| [Embedded Statuses](pipelineruns.md#configuring-usage-of-taskrun-and-run-embedded-statuses) | [TEP-0100](https://github.com/tektoncd/community/blob/main/teps/0100-embedded-taskruns-and-runs-status-in-pipelineruns.md) | | |
429+
| [Task-level Resource Requirements](compute-resources.md#task-level-compute-resources-configuration) | [TEP-0104](https://github.com/tektoncd/community/blob/main/teps/0104-tasklevel-resource-requirements.md) | | |
429430

430431
## Configuring High Availability
431432

‎docs/pipelineruns.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ weight: 500
1414
- [Tekton Bundles](#tekton-bundles)
1515
- [Remote Pipelines](#remote-pipelines)
1616
- [Specifying <code>Resources</code>](#specifying-resources)
17+
- [Specifying Task-level `ComputeResources`](#specifying-task-level-computeresources)
1718
- [Specifying <code>Parameters</code>](#specifying-parameters)
1819
- [Propagated Parameters](#propagated-parameters)
1920
- [Scope and Precedence](#scope-and-precedence)
@@ -260,6 +261,40 @@ spec:
260261
until their respective `Pods` or the entire `PipelineRun` are deleted. This also applies
261262
to all `persistentVolumeClaims` generated internally.
262263

264+
### Specifying Task-level `ComputeResources`
265+
266+
**([alpha only](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#alpha-features))**
267+
(This feature is under development and not functional yet. Stay tuned!)
268+
269+
Task-level compute resources can be configured in `PipelineRun.TaskRunSpecs.ComputeResources` or `TaskRun.ComputeResources`.
270+
271+
e.g.
272+
273+
```yaml
274+
apiVersion: tekton.dev/v1beta1
275+
kind: Pipeline
276+
metadata:
277+
name: pipeline
278+
spec:
279+
tasks:
280+
- name: task
281+
---
282+
apiVersion: tekton.dev/v1beta1
283+
kind: PipelineRun
284+
metadata:
285+
name: pipelinerun
286+
spec:
287+
pipelineRef:
288+
name: pipeline
289+
taskRunSpecs:
290+
- pipelineTaskName: task
291+
computeResources:
292+
requests:
293+
cpu: 2
294+
```
295+
296+
Further details and examples could be found in [Compute Resources in Tekton](https://github.com/tektoncd/pipeline/blob/main/docs/compute-resources.md).
297+
263298
### Specifying `Parameters`
264299

265300
(See also [Specifying Parameters in Tasks](tasks.md#specifying-parameters))

‎docs/taskruns.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ weight: 300
1818
- [Extra Parameters](#extra-parameters)
1919
- [Specifying `Resources`](#specifying-resources)
2020
- [Specifying `Resource` limits](#specifying-resource-limits)
21+
- [Specifying Task-level `ComputeResources`](#specifying-task-level-computeresources)
2122
- [Specifying a `Pod` template](#specifying-a-pod-template)
2223
- [Specifying `Workspaces`](#specifying-workspaces)
2324
- [Specifying `Sidecars`](#specifying-sidecars)
@@ -323,6 +324,40 @@ Each Step in a Task can specify its resource requirements. See
323324
[Defining `Steps`](tasks.md#defining-steps). Resource requirements defined in Steps and Sidecars
324325
may be overridden by a TaskRun's StepOverrides and SidecarOverrides.
325326

327+
### Specifying Task-level `ComputeResources`
328+
329+
**([alpha only](https://github.com/tektoncd/pipeline/blob/main/docs/install.md#alpha-features))**
330+
(This feature is under development and not functional yet. Stay tuned!)
331+
332+
Task-level compute resources can be configured in `TaskRun.ComputeResources`, or `PipelineRun.TaskRunSpecs.ComputeResources`.
333+
334+
e.g.
335+
336+
```yaml
337+
apiVersion: tekton.dev/v1beta1
338+
kind: Task
339+
metadata:
340+
name: task
341+
spec:
342+
steps:
343+
- name: foo
344+
---
345+
apiVersion: tekton.dev/v1beta1
346+
kind: TaskRun
347+
metadata:
348+
name: taskrun
349+
spec:
350+
taskRef:
351+
name: task
352+
computeResources:
353+
requests:
354+
cpu: 1
355+
limits:
356+
cpu: 2
357+
```
358+
359+
Further details and examples could be found in [Compute Resources in Tekton](https://github.com/tektoncd/pipeline/blob/main/docs/compute-resources.md).
360+
326361
### Specifying a `Pod` template
327362

328363
You can specify a [`Pod` template](podtemplates.md) configuration that will serve as the configuration starting

‎pkg/apis/pipeline/v1beta1/openapi_generated.go‎

Lines changed: 14 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎pkg/apis/pipeline/v1beta1/pipelinerun_types.go‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ import (
2020
"context"
2121
"time"
2222

23+
corev1 "k8s.io/api/core/v1"
2324
"k8s.io/apimachinery/pkg/runtime"
2425

2526
"github.com/tektoncd/pipeline/pkg/apis/config"
@@ -540,6 +541,9 @@ type PipelineTaskRunSpec struct {
540541

541542
// +optional
542543
Metadata *PipelineTaskMetadata `json:"metadata,omitempty"`
544+
545+
// Compute resources to use for this TaskRun
546+
ComputeResources *corev1.ResourceRequirements `json:"computeResources,omitempty"`
543547
}
544548

545549
// GetTaskRunSpec returns the task specific spec for a given

‎pkg/apis/pipeline/v1beta1/pipelinerun_validation.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -195,6 +195,9 @@ func validateTaskRunSpec(ctx context.Context, trs PipelineTaskRunSpec) (errs *ap
195195
if trs.SidecarOverrides != nil {
196196
errs = errs.Also(validateSidecarOverrides(trs.SidecarOverrides).ViaField("sidecarOverrides"))
197197
}
198+
if trs.ComputeResources != nil {
199+
errs = errs.Also(validateTaskRunComputeResources(trs.ComputeResources, trs.StepOverrides))
200+
}
198201
} else {
199202
if trs.StepOverrides != nil {
200203
errs = errs.Also(apis.ErrDisallowedFields("stepOverrides"))

‎pkg/apis/pipeline/v1beta1/pipelinerun_validation_test.go‎

Lines changed: 72 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -610,7 +610,32 @@ func TestPipelineRunSpec_Invalidate(t *testing.T) {
610610
},
611611
wantErr: apis.ErrMissingField("taskRunSpecs[0].sidecarOverrides[0].name"),
612612
withContext: enableAlphaAPIFields,
613+
}, {
614+
name: "invalid both step-level (stepOverrides.resources) and task-level (taskRunSpecs.resources) resource requirements configured",
615+
spec: v1beta1.PipelineRunSpec{
616+
PipelineRef: &v1beta1.PipelineRef{Name: "pipeline"},
617+
TaskRunSpecs: []v1beta1.PipelineTaskRunSpec{
618+
{
619+
PipelineTaskName: "pipelineTask",
620+
StepOverrides: []v1beta1.TaskRunStepOverride{{
621+
Name: "stepOverride",
622+
Resources: corev1.ResourceRequirements{
623+
Requests: corev1.ResourceList{corev1.ResourceMemory: corev1resources.MustParse("1Gi")},
624+
}},
625+
},
626+
ComputeResources: &corev1.ResourceRequirements{
627+
Requests: corev1.ResourceList{corev1.ResourceMemory: corev1resources.MustParse("2Gi")},
628+
},
629+
},
630+
},
631+
},
632+
wantErr: apis.ErrMultipleOneOf(
633+
"taskRunSpecs[0].stepOverrides.resources",
634+
"taskRunSpecs[0].computeResources",
635+
),
636+
withContext: enableAlphaAPIFields,
613637
}}
638+
614639
for _, ps := range tests {
615640
t.Run(ps.name, func(t *testing.T) {
616641
ctx := context.Background()
@@ -627,8 +652,9 @@ func TestPipelineRunSpec_Invalidate(t *testing.T) {
627652

628653
func TestPipelineRunSpec_Validate(t *testing.T) {
629654
tests := []struct {
630-
name string
631-
spec v1beta1.PipelineRunSpec
655+
name string
656+
spec v1beta1.PipelineRunSpec
657+
withContext func(context.Context) context.Context
632658
}{{
633659
name: "PipelineRun without pipelineRef",
634660
spec: v1beta1.PipelineRunSpec{
@@ -641,10 +667,53 @@ func TestPipelineRunSpec_Validate(t *testing.T) {
641667
}},
642668
},
643669
},
670+
}, {
671+
name: "valid task-level (taskRunSpecs.resources) resource requirements configured",
672+
spec: v1beta1.PipelineRunSpec{
673+
PipelineRef: &v1beta1.PipelineRef{Name: "pipeline"},
674+
TaskRunSpecs: []v1beta1.PipelineTaskRunSpec{{
675+
PipelineTaskName: "pipelineTask",
676+
StepOverrides: []v1beta1.TaskRunStepOverride{{
677+
Name: "stepOverride",
678+
}},
679+
ComputeResources: &corev1.ResourceRequirements{
680+
Requests: corev1.ResourceList{corev1.ResourceMemory: corev1resources.MustParse("2Gi")},
681+
},
682+
}},
683+
},
684+
withContext: enableAlphaAPIFields,
685+
}, {
686+
name: "valid sidecar and task-level (taskRunSpecs.resources) resource requirements configured",
687+
spec: v1beta1.PipelineRunSpec{
688+
PipelineRef: &v1beta1.PipelineRef{Name: "pipeline"},
689+
TaskRunSpecs: []v1beta1.PipelineTaskRunSpec{{
690+
PipelineTaskName: "pipelineTask",
691+
StepOverrides: []v1beta1.TaskRunStepOverride{{
692+
Name: "stepOverride",
693+
}},
694+
ComputeResources: &corev1.ResourceRequirements{
695+
Requests: corev1.ResourceList{corev1.ResourceMemory: corev1resources.MustParse("2Gi")},
696+
},
697+
SidecarOverrides: []v1beta1.TaskRunSidecarOverride{{
698+
Name: "sidecar",
699+
Resources: corev1.ResourceRequirements{
700+
Requests: corev1.ResourceList{
701+
corev1.ResourceMemory: corev1resources.MustParse("4Gi"),
702+
},
703+
},
704+
}},
705+
}},
706+
},
707+
withContext: enableAlphaAPIFields,
644708
}}
709+
645710
for _, ps := range tests {
646711
t.Run(ps.name, func(t *testing.T) {
647-
if err := ps.spec.Validate(context.Background()); err != nil {
712+
ctx := context.Background()
713+
if ps.withContext != nil {
714+
ctx = ps.withContext(ctx)
715+
}
716+
if err := ps.spec.Validate(ctx); err != nil {
648717
t.Error(err)
649718
}
650719
})

‎pkg/apis/pipeline/v1beta1/swagger.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1394,6 +1394,10 @@
13941394
"description": "PipelineTaskRunSpec can be used to configure specific specs for a concrete Task",
13951395
"type": "object",
13961396
"properties": {
1397+
"computeResources": {
1398+
"description": "Compute resources to use for this TaskRun",
1399+
"$ref": "#/definitions/v1.ResourceRequirements"
1400+
},
13971401
"metadata": {
13981402
"$ref": "#/definitions/v1beta1.PipelineTaskMetadata"
13991403
},
@@ -2433,6 +2437,10 @@
24332437
"description": "TaskRunSpec defines the desired state of TaskRun",
24342438
"type": "object",
24352439
"properties": {
2440+
"computeResources": {
2441+
"description": "Compute resources to use for this TaskRun",
2442+
"$ref": "#/definitions/v1.ResourceRequirements"
2443+
},
24362444
"debug": {
24372445
"$ref": "#/definitions/v1beta1.TaskRunDebug"
24382446
},

0 commit comments

Comments
 (0)