feat(pipeline): allow variable substitution in pipeline.tasks[].onError #8600

l-qing · 2025-02-27T15:59:07Z

In the reconcile logic of PipelineRun, add the ability to replace the pipeline.tasks[].onError field. At the same time, the validation of pipelineSpec legality has been delayed.

Changes

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
Has Tests included if any functionality added or changed
pre-commit Passed
Follows the commit message standard
Meets the Tekton contributor standards (including functionality, content, code)
Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

feat(pipeline): allow variable substitution in pipeline.tasks[].onError

/kind feature

tekton-robot · 2025-02-27T16:09:28Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.3%	99.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.7%	91.6%	-0.1
pkg/reconciler/pipelinerun/resources/apply.go	97.3%	98.3%	0.9

l-qing · 2025-02-27T16:04:17Z

docs/variables.md

 | `TaskRun`     | `spec.workspaces[].csi.driver`                                  |
 | `TaskRun`     | `spec.workspaces[].csi.nodePublishSecretRef.name`               |
 | `Pipeline`    | `spec.tasks[].params[].value`                                   |
-| `Pipeline`    | `spec.tasks[].conditions[].params[].value`                      |


The replacement of spec.tasks[].conditions[].params[].value is a capability of v1alpha1, which no longer has that field.

The current complete definition of the pipelinetask structure is as follows:

pipeline/pkg/apis/pipeline/v1/pipeline_types.go

Lines 179 to 257 in aec465e

// PipelineTask defines a task in a Pipeline, passing inputs from both

// Params and from the output of previous tasks.

type PipelineTask struct {

// Name is the name of this task within the context of a Pipeline. Name is

// used as a coordinate with the `from` and `runAfter` fields to establish

// the execution order of tasks relative to one another.

Name string `json:"name,omitempty"`

// DisplayName is the display name of this task within the context of a Pipeline.

// This display name may be used to populate a UI.

// +optional

DisplayName string `json:"displayName,omitempty"`

// Description is the description of this task within the context of a Pipeline.

// This description may be used to populate a UI.

// +optional

Description string `json:"description,omitempty"`

// TaskRef is a reference to a task definition.

// +optional

TaskRef *TaskRef `json:"taskRef,omitempty"`

// TaskSpec is a specification of a task

// Specifying TaskSpec can be disabled by setting

// `disable-inline-spec` feature flag..

// +optional

TaskSpec *EmbeddedTask `json:"taskSpec,omitempty"`

// When is a list of when expressions that need to be true for the task to run

// +optional

When WhenExpressions `json:"when,omitempty"`

// Retries represents how many times this task should be retried in case of task failure: ConditionSucceeded set to False

// +optional

Retries int `json:"retries,omitempty"`

// RunAfter is the list of PipelineTask names that should be executed before

// this Task executes. (Used to force a specific ordering in graph execution.)

// +optional

// +listType=atomic

RunAfter []string `json:"runAfter,omitempty"`

// Parameters declares parameters passed to this task.

// +optional

// +listType=atomic

Params Params `json:"params,omitempty"`

// Matrix declares parameters used to fan out this task.

// +optional

Matrix *Matrix `json:"matrix,omitempty"`

// Workspaces maps workspaces from the pipeline spec to the workspaces

// declared in the Task.

// +optional

// +listType=atomic

Workspaces []WorkspacePipelineTaskBinding `json:"workspaces,omitempty"`

// Time after which the TaskRun times out. Defaults to 1 hour.

// Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration

// +optional

Timeout *metav1.Duration `json:"timeout,omitempty"`

// PipelineRef is a reference to a pipeline definition

// Note: PipelineRef is in preview mode and not yet supported

// +optional

PipelineRef *PipelineRef `json:"pipelineRef,omitempty"`

// PipelineSpec is a specification of a pipeline

// Note: PipelineSpec is in preview mode and not yet supported

// Specifying PipelineSpec can be disabled by setting

// `disable-inline-spec` feature flag..

// +optional

PipelineSpec *PipelineSpec `json:"pipelineSpec,omitempty"`

// OnError defines the exiting behavior of a PipelineRun on error

// can be set to [ continue | stopAndFail ]

// +optional

OnError PipelineTaskOnErrorType `json:"onError,omitempty"`

}

The previous code for replacing variables in PipelineTask is as follows:

pipeline/pkg/reconciler/pipelinerun/resources/apply.go

Lines 301 to 352 in aec465e

// ApplyReplacements replaces placeholders for declared parameters with the specified replacements.

func ApplyReplacements(p *v1.PipelineSpec, replacements map[string]string, arrayReplacements map[string][]string, objectReplacements map[string]map[string]string) *v1.PipelineSpec {

p = p.DeepCopy()

for i := range p.Tasks {

p.Tasks[i].Params = p.Tasks[i].Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)

if p.Tasks[i].IsMatrixed() {

p.Tasks[i].Matrix.Params = p.Tasks[i].Matrix.Params.ReplaceVariables(replacements, arrayReplacements, nil)

for j := range p.Tasks[i].Matrix.Include {

p.Tasks[i].Matrix.Include[j].Params = p.Tasks[i].Matrix.Include[j].Params.ReplaceVariables(replacements, nil, nil)

}

} else {

p.Tasks[i].DisplayName = substitution.ApplyReplacements(p.Tasks[i].DisplayName, replacements)

}

for j := range p.Tasks[i].Workspaces {

p.Tasks[i].Workspaces[j].SubPath = substitution.ApplyReplacements(p.Tasks[i].Workspaces[j].SubPath, replacements)

}

p.Tasks[i].When = p.Tasks[i].When.ReplaceVariables(replacements, arrayReplacements)

if p.Tasks[i].TaskRef != nil {

if p.Tasks[i].TaskRef.Params != nil {

p.Tasks[i].TaskRef.Params = p.Tasks[i].TaskRef.Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)

}

p.Tasks[i].TaskRef.Name = substitution.ApplyReplacements(p.Tasks[i].TaskRef.Name, replacements)

}

p.Tasks[i] = propagateParams(p.Tasks[i], replacements, arrayReplacements, objectReplacements)

}

for i := range p.Finally {

p.Finally[i].Params = p.Finally[i].Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)

if p.Finally[i].IsMatrixed() {

p.Finally[i].Matrix.Params = p.Finally[i].Matrix.Params.ReplaceVariables(replacements, arrayReplacements, nil)

for j := range p.Finally[i].Matrix.Include {

p.Finally[i].Matrix.Include[j].Params = p.Finally[i].Matrix.Include[j].Params.ReplaceVariables(replacements, nil, nil)

}

} else {

p.Finally[i].DisplayName = substitution.ApplyReplacements(p.Finally[i].DisplayName, replacements)

}

for j := range p.Finally[i].Workspaces {

p.Finally[i].Workspaces[j].SubPath = substitution.ApplyReplacements(p.Finally[i].Workspaces[j].SubPath, replacements)

}

p.Finally[i].When = p.Finally[i].When.ReplaceVariables(replacements, arrayReplacements)

if p.Finally[i].TaskRef != nil {

if p.Finally[i].TaskRef.Params != nil {

p.Finally[i].TaskRef.Params = p.Finally[i].TaskRef.Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)

}

p.Finally[i].TaskRef.Name = substitution.ApplyReplacements(p.Finally[i].TaskRef.Name, replacements)

}

p.Finally[i] = propagateParams(p.Finally[i], replacements, arrayReplacements, objectReplacements)

}

return p

}

l-qing · 2025-02-27T16:05:01Z

pkg/apis/pipeline/v1/pipeline_validation.go

 }

+// ValidateOnError validates the OnError field of a PipelineTask
+func (pt PipelineTask) ValidateOnError(ctx context.Context) (errs *apis.FieldError) {


It's convenient to reuse this verification logic in the reconcile logic of PipelineRun.

l-qing · 2025-02-27T16:12:17Z

pkg/reconciler/pipelinerun/pipelinerun.go

 	pr.Status.PipelineSpec = pipelineSpec

+	// validate pipelineSpec after apply parameters
+	if err := validatePipelineSpecAfterApplyParameters(ctx, pipelineSpec); err != nil {


This is what is called deferred validation.
Targeted validation is performed after applying parameters.
The reason for not calling pipelineSpec.Validate again is that many field validations are unnecessary.

pipeline/pkg/reconciler/pipelinerun/pipelinerun.go

Lines 481 to 487 in aec465e

if err := pipelineSpec.Validate(ctx); err != nil {

// This Run has failed, so we need to mark it as failed and stop reconciling it

pr.Status.MarkFailed(v1.PipelineRunReasonFailedValidation.String(),

"Pipeline %s/%s can't be Run; it has an invalid spec: %s",

pipelineMeta.Namespace, pipelineMeta.Name, pipelineErrors.WrapUserError(err))

return controller.NewPermanentError(err)

}

l-qing · 2025-02-27T16:16:12Z

pkg/reconciler/pipelinerun/resources/apply.go

-			for j := range p.Tasks[i].Matrix.Include {
-				p.Tasks[i].Matrix.Include[j].Params = p.Tasks[i].Matrix.Include[j].Params.ReplaceVariables(replacements, nil, nil)
+// replaceVariablesInPipelineTasks handles variable replacement for a slice of PipelineTasks in-place
+func replaceVariablesInPipelineTasks(tasks []v1.PipelineTask, replacements map[string]string,


The previous code contained a lot of duplication; here it has been simply refactored.

l-qing · 2025-02-27T16:18:32Z

pkg/reconciler/pipelinerun/resources/apply.go

+			tasks[i].TaskRef.Name = substitution.ApplyReplacements(tasks[i].TaskRef.Name, replacements)
 		}
-		p.Tasks[i] = propagateParams(p.Tasks[i], replacements, arrayReplacements, objectReplacements)
+		tasks[i].OnError = v1.PipelineTaskOnErrorType(substitution.ApplyReplacements(string(tasks[i].OnError), replacements))


The most important line of code in this new update is the support for variable replacement logic in OnError. 😆

tekton-robot · 2025-02-27T16:31:03Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.3%	99.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.7%	91.6%	-0.1
pkg/reconciler/pipelinerun/resources/apply.go	97.3%	98.3%	0.9

tekton-robot · 2025-03-12T02:30:56Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.3%	99.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.7%	91.6%	-0.1
pkg/reconciler/pipelinerun/resources/apply.go	97.3%	98.3%	0.9

vdemeester

👍🏼

tekton-robot · 2025-03-12T15:00:26Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [vdemeester]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

fix tektoncd#8564 In the reconcile logic of PipelineRun, add the ability to replace the `pipeline.tasks[].onError` field. At the same time, the validation of pipelineSpec legality has been delayed.

tekton-robot · 2025-03-14T08:24:51Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage-df to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/apis/pipeline/v1/pipeline_validation.go	99.3%	99.3%	0.0
pkg/reconciler/pipelinerun/pipelinerun.go	91.7%	91.6%	-0.1
pkg/reconciler/pipelinerun/resources/apply.go	97.3%	98.3%	0.9

waveywaves · 2025-04-07T10:22:59Z

/lgtm

vdemeester · 2025-07-29T12:20:40Z

Arf, this is not implemented for v1beta1, at least the validation. We need to fix this.

l-qing · 2025-07-29T16:24:50Z

Arf, this is not implemented for v1beta1, at least the validation. We need to fix this.

Okay, I’ll take a look tomorrow.

vdemeester · 2025-07-30T08:11:37Z

Arf, this is not implemented for v1beta1, at least the validation. We need to fix this.

Okay, I’ll take a look tomorrow.

Thanks 🤗

github-project-automation bot added this to Tekton Community Roadmap Feb 27, 2025

github-project-automation bot moved this to Todo in Tekton Community Roadmap Feb 27, 2025

tekton-robot added kind/feature Categorizes issue or PR as related to a new feature. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Feb 27, 2025

tekton-robot requested review from afrittoli and pritidesai February 27, 2025 15:59

tekton-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Feb 27, 2025

l-qing commented Feb 27, 2025

View reviewed changes

l-qing force-pushed the feat/variable-substitution-onError branch from f62cd92 to aedd707 Compare February 27, 2025 16:20

l-qing force-pushed the feat/variable-substitution-onError branch from aedd707 to 9fa4f9f Compare March 12, 2025 02:06

vdemeester approved these changes Mar 12, 2025

View reviewed changes

tekton-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 12, 2025

feat(pipeline): allow variable substitution in pipeline.tasks[].onError

fa78954

fix tektoncd#8564 In the reconcile logic of PipelineRun, add the ability to replace the `pipeline.tasks[].onError` field. At the same time, the validation of pipelineSpec legality has been delayed.

l-qing force-pushed the feat/variable-substitution-onError branch from 9fa4f9f to fa78954 Compare March 14, 2025 04:24

l-qing closed this Mar 14, 2025

github-project-automation bot moved this from Todo to Done in Tekton Community Roadmap Mar 14, 2025

l-qing reopened this Mar 14, 2025

github-project-automation bot moved this from Done to In Progress in Tekton Community Roadmap Mar 14, 2025

tekton-robot assigned waveywaves Apr 7, 2025

tekton-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 7, 2025

tekton-robot merged commit 5b050cd into tektoncd:main Apr 7, 2025
33 checks passed

github-project-automation bot moved this from In Progress to Done in Tekton Community Roadmap Apr 7, 2025

l-qing mentioned this pull request Jul 30, 2025

fix(pipeline): support variables in onError for pipeline v1beta1 #8931

Merged

8 tasks

	// PipelineTask defines a task in a Pipeline, passing inputs from both
	// Params and from the output of previous tasks.
	type PipelineTask struct {
	// Name is the name of this task within the context of a Pipeline. Name is
	// used as a coordinate with the `from` and `runAfter` fields to establish
	// the execution order of tasks relative to one another.
	Name string `json:"name,omitempty"`

	// DisplayName is the display name of this task within the context of a Pipeline.
	// This display name may be used to populate a UI.
	// +optional
	DisplayName string `json:"displayName,omitempty"`

	// Description is the description of this task within the context of a Pipeline.
	// This description may be used to populate a UI.
	// +optional
	Description string `json:"description,omitempty"`

	// TaskRef is a reference to a task definition.
	// +optional
	TaskRef *TaskRef `json:"taskRef,omitempty"`

	// TaskSpec is a specification of a task
	// Specifying TaskSpec can be disabled by setting
	// `disable-inline-spec` feature flag..
	// +optional
	TaskSpec *EmbeddedTask `json:"taskSpec,omitempty"`

	// When is a list of when expressions that need to be true for the task to run
	// +optional
	When WhenExpressions `json:"when,omitempty"`

	// Retries represents how many times this task should be retried in case of task failure: ConditionSucceeded set to False
	// +optional
	Retries int `json:"retries,omitempty"`

	// RunAfter is the list of PipelineTask names that should be executed before
	// this Task executes. (Used to force a specific ordering in graph execution.)
	// +optional
	// +listType=atomic
	RunAfter []string `json:"runAfter,omitempty"`

	// Parameters declares parameters passed to this task.
	// +optional
	// +listType=atomic
	Params Params `json:"params,omitempty"`

	// Matrix declares parameters used to fan out this task.
	// +optional
	Matrix *Matrix `json:"matrix,omitempty"`

	// Workspaces maps workspaces from the pipeline spec to the workspaces
	// declared in the Task.
	// +optional
	// +listType=atomic
	Workspaces []WorkspacePipelineTaskBinding `json:"workspaces,omitempty"`

	// Time after which the TaskRun times out. Defaults to 1 hour.
	// Refer Go's ParseDuration documentation for expected format: https://golang.org/pkg/time/#ParseDuration
	// +optional
	Timeout *metav1.Duration `json:"timeout,omitempty"`

	// PipelineRef is a reference to a pipeline definition
	// Note: PipelineRef is in preview mode and not yet supported
	// +optional
	PipelineRef *PipelineRef `json:"pipelineRef,omitempty"`

	// PipelineSpec is a specification of a pipeline
	// Note: PipelineSpec is in preview mode and not yet supported
	// Specifying PipelineSpec can be disabled by setting
	// `disable-inline-spec` feature flag..
	// +optional
	PipelineSpec *PipelineSpec `json:"pipelineSpec,omitempty"`

	// OnError defines the exiting behavior of a PipelineRun on error
	// can be set to [ continue \| stopAndFail ]
	// +optional
	OnError PipelineTaskOnErrorType `json:"onError,omitempty"`
	}

	// ApplyReplacements replaces placeholders for declared parameters with the specified replacements.
	func ApplyReplacements(p v1.PipelineSpec, replacements map[string]string, arrayReplacements map[string][]string, objectReplacements map[string]map[string]string) v1.PipelineSpec {
	p = p.DeepCopy()

	for i := range p.Tasks {
	p.Tasks[i].Params = p.Tasks[i].Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)
	if p.Tasks[i].IsMatrixed() {
	p.Tasks[i].Matrix.Params = p.Tasks[i].Matrix.Params.ReplaceVariables(replacements, arrayReplacements, nil)
	for j := range p.Tasks[i].Matrix.Include {
	p.Tasks[i].Matrix.Include[j].Params = p.Tasks[i].Matrix.Include[j].Params.ReplaceVariables(replacements, nil, nil)
	}
	} else {
	p.Tasks[i].DisplayName = substitution.ApplyReplacements(p.Tasks[i].DisplayName, replacements)
	}
	for j := range p.Tasks[i].Workspaces {
	p.Tasks[i].Workspaces[j].SubPath = substitution.ApplyReplacements(p.Tasks[i].Workspaces[j].SubPath, replacements)
	}
	p.Tasks[i].When = p.Tasks[i].When.ReplaceVariables(replacements, arrayReplacements)
	if p.Tasks[i].TaskRef != nil {
	if p.Tasks[i].TaskRef.Params != nil {
	p.Tasks[i].TaskRef.Params = p.Tasks[i].TaskRef.Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)
	}
	p.Tasks[i].TaskRef.Name = substitution.ApplyReplacements(p.Tasks[i].TaskRef.Name, replacements)
	}
	p.Tasks[i] = propagateParams(p.Tasks[i], replacements, arrayReplacements, objectReplacements)
	}

	for i := range p.Finally {
	p.Finally[i].Params = p.Finally[i].Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)
	if p.Finally[i].IsMatrixed() {
	p.Finally[i].Matrix.Params = p.Finally[i].Matrix.Params.ReplaceVariables(replacements, arrayReplacements, nil)
	for j := range p.Finally[i].Matrix.Include {
	p.Finally[i].Matrix.Include[j].Params = p.Finally[i].Matrix.Include[j].Params.ReplaceVariables(replacements, nil, nil)
	}
	} else {
	p.Finally[i].DisplayName = substitution.ApplyReplacements(p.Finally[i].DisplayName, replacements)
	}
	for j := range p.Finally[i].Workspaces {
	p.Finally[i].Workspaces[j].SubPath = substitution.ApplyReplacements(p.Finally[i].Workspaces[j].SubPath, replacements)
	}
	p.Finally[i].When = p.Finally[i].When.ReplaceVariables(replacements, arrayReplacements)
	if p.Finally[i].TaskRef != nil {
	if p.Finally[i].TaskRef.Params != nil {
	p.Finally[i].TaskRef.Params = p.Finally[i].TaskRef.Params.ReplaceVariables(replacements, arrayReplacements, objectReplacements)
	}
	p.Finally[i].TaskRef.Name = substitution.ApplyReplacements(p.Finally[i].TaskRef.Name, replacements)
	}
	p.Finally[i] = propagateParams(p.Finally[i], replacements, arrayReplacements, objectReplacements)
	}

	return p
	}

	if err := pipelineSpec.Validate(ctx); err != nil {
	// This Run has failed, so we need to mark it as failed and stop reconciling it
	pr.Status.MarkFailed(v1.PipelineRunReasonFailedValidation.String(),
	"Pipeline %s/%s can't be Run; it has an invalid spec: %s",
	pipelineMeta.Namespace, pipelineMeta.Name, pipelineErrors.WrapUserError(err))
	return controller.NewPermanentError(err)
	}

feat(pipeline): allow variable substitution in pipeline.tasks[].onError #8600

feat(pipeline): allow variable substitution in pipeline.tasks[].onError #8600

Uh oh!

Conversation

l-qing commented Feb 27, 2025

Changes

Submitter Checklist

Release Notes

Uh oh!

tekton-robot commented Feb 27, 2025

Uh oh!

l-qing Feb 27, 2025

Choose a reason for hiding this comment

Uh oh!

l-qing Feb 27, 2025

Choose a reason for hiding this comment

Uh oh!

l-qing Feb 27, 2025

Choose a reason for hiding this comment

Uh oh!

l-qing Feb 27, 2025

Choose a reason for hiding this comment

Uh oh!

l-qing Feb 27, 2025

Choose a reason for hiding this comment

Uh oh!

tekton-robot commented Feb 27, 2025

Uh oh!

tekton-robot commented Mar 12, 2025

Uh oh!

vdemeester left a comment

Choose a reason for hiding this comment

Uh oh!

tekton-robot commented Mar 12, 2025

Uh oh!

tekton-robot commented Mar 14, 2025

Uh oh!

waveywaves commented Apr 7, 2025

Uh oh!

Uh oh!

vdemeester commented Jul 29, 2025

Uh oh!

l-qing commented Jul 29, 2025

Uh oh!

vdemeester commented Jul 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants