TEP-0090: Matrix - Failure Strategies

[jerop]

TEP-0090: Matrix proposed executing a PipelineTask in
parallel TaskRuns and Runs with substitutions from combinations
of Parameters in a Matrix.

In this change, we propose a fail-fast stategy for handling failures
in TaskRuns and Runs created from a Matrix. This makes it easier
to control the execution of the many TaskRuns or Runs created from
a Matrix - up to 256.

This approach also aligns with the default behavior of Matrix in
other Continuous Delivery systems e.g GitHub Actions.

If needed, we can explore supporting other failure strategies later.

/kind tep

[jerop]

cc @abayer @dibyom @lbernick @pritidesai please take a look

[abayer]

This is the right call - in Jenkins Pipeline, we inherited certain assumptions around failure behavior for parallel executions (both in a matrix and not) which resulted in us having to support making fail-fast configurable, and I don't think it was worth doing that in the end. When we're starting from scratch, like here, we have the opportunity to say "this is how things work" and adjust if sufficient demand comes up in the future, rather than prematurely optimizing. So, yeah. 👍

[tekton-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abayer, dibyom, lbernick

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• teps/OWNERS [dibyom,lbernick]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[pritidesai]

thanks @jerop, looking for more clarification, how are we failing already running taskRuns or runs from the same pipelineTask? Sending termination signal?

Can we implement something like stopping mode by default, the similar approach we take when a task in a pipeline fails?

[pritidesai]

the reason I am asking is I am assuming stopping is easier to implement 🤣 And does not result in partial execution ...

[jerop]

@pritidesai yes, similarly to how we cancel TaskRuns for example - it may be easier to implement stopping mode but wondering if that's the behavior we want for Matrix separate from the implementation - maybe partial execution could be the reason not to do this?

[pritidesai]

stopping will be consistent to how we address failure of a non-matrix pipelineTask.

Just an example, if building an image of one application fails, I do not want that failure to stop building images of 20 other applications.

[jerop]

makes sense - added this to the API WG agenda on Monday so we can decide on the best way forward

[lbernick]

/assign @abayer @lbernick @pritidesai @dibyom

[jerop]

Discussed at the API WG on 06/13 and we agreed on the following:

• There are use cases for both terminating early (failing fast) and allowing completing execution of running taskruns (stopping mode)
• Terminating early upon failure is applies more broadly beyond matrix; this is something we can address separately for all taskruns
• The initial release of matrix will implement the "stopping mode" that the rest of the pipeline uses; we will revisit failure strategies later if needed

/close

[tekton-robot]

@jerop: Closed this PR.

In response to this:

Discussed at the API WG on 06/13 and we agreed on the following:

• There are use cases for both terminating early (failing fast) and allowing completing execution of running taskruns (stopping mode)
• Terminating early upon failure is applies more broadly beyond matrix; this is something we can address separately for all taskruns
• The initial release of matrix will implement the "stopping mode" that the rest of the pipeline uses; we will revisit failure strategies later if needed

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.