Pipeline failing with "invalid pipelineresults" when referencing results of skipped tasks

[thomascube]

Expected Behavior

According to the documentation pipeline results can contain references to task results. If a task did not run because of it's whenConditions, the pipeline result is not emitted but the pipeline succeeds. This worked with v0.37.5 of Tekton Pipelines available on OpenShift.

Actual Behavior

With the update of the RedHat OpenShift Pipelines Operator, which brings Tekton Pipeline v0.41.0, pipelines with skipped tasks and composed results now fail with the message invalid pipelineresults [result-goodbye], the referred results don't exist.

Steps to Reproduce the Problem

Run the following PipelineRun:

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: pipelinerun-test
spec:
  pipelineSpec:
    params:
      - name: say-hello
        default: 'false'
    tasks:
      - name: hello
        taskSpec:
          results:
            - name: result-one
          steps:
          - image: alpine
            script: |
              #!/bin/sh
              echo "Hello world!"
              echo -n "RES1" > $(results.result-one.path)
  
      - name: goodbye
        runAfter:
          - hello
        taskSpec:
          results:
            - name: result-two
          steps:
          - image: alpine
            script: |
              #!/bin/sh
              echo "Goodbye world!"
              echo -n "RES2" > $(results.result-two.path)
        when:
          - input: $(params.say-hello)
            operator: in
            values: ["true"]

    results:
      - name: result-hello
        description: Result one
        value: '$(tasks.hello.results.result-one)'
      - name: result-goodbye
        description: Result two
        value: '$(tasks.goodbye.results.result-two)'

Additional Info

• Kubernetes version:

Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.4+a34b9e9", GitCommit:"b6d1f054747e9886f61dd85316deac3415e2726f", GitTreeState:"clean", BuildDate:"2023-01-10T15:55:28Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}

• Tekton Pipeline version:

Pipeline version: v0.41.0
Triggers version: v0.22.0
Openshift Pipelines Version: 1.9.0

[thomascube]

Success on v0.37.5:

Failure on v0.41.0:

[Yongxuanzhang]

Hi @thomascube, thanks!
I think the validation is added intentionally, do you think it makes more sense to remove the check?
Since I may lack the knowledge of the use cases, I'd like to learn more about your opinons.

[kyubisation]

@Yongxuanzhang We would like to display these values in our UI tool (if they are available). Is there any way to default to empty, if the task was not run?
e.g. something like '$(tasks.hello.results.result-one || '')' or '$(tasks.hello && tasks.hello.results.result-one)'?

[jerop]

@kyubisation we don't have default result yet but it's something that @vinamra28 has been looking into -- considering a syntax similar to the first one you suggested: '$(tasks.hello.results.result-one || '')'

[Yongxuanzhang]

@Yongxuanzhang We would like to display these values in our UI tool (if they are available). Is there any way to default to empty, if the task was not run? e.g. something like '$(tasks.hello.results.result-one || '')' or '$(tasks.hello && tasks.hello.results.result-one)'?

I think default results would be a solution. But it may take much longer time?

I wonder before this update, if the results are not available, there will be nothing showing in UI?
The reason I add the check is to help debugging. So you would know which results are not emitted.
I'm thinking if there is a better way to meet both needs?
e.g. We don't fail the pipelinerun but only log the warning message?

Or actually we don't want to have any check/validation on this? If so I think we could just remove the validation.

@kyubisation @jerop any suggestions?

[thomascube]

@Yongxuanzhang I think the old behavior which is also described in the documentation under "A Pipeline Result is not emitted if any of the following are true" makes perfectly sense and we were running fine with it. With the new check and the failure of the pipeline it's impossible to use task results in conjunction with when conditions.

[kyubisation]

@Yongxuanzhang Having validation is reasonable, but directly breaking the pipeline without a workaround/migration possibility is not ideal. I'd prefer to have this either be opt-in or have to have a fallback/default available at the same time.

[Yongxuanzhang]

@Yongxuanzhang I think the old behavior which is also described in the documentation under "A Pipeline Result is not emitted if any of the following are true" makes perfectly sense and we were running fine with it. With the new check and the failure of the pipeline it's impossible to use task results in conjunction with when conditions.

In the doc you linked it says:

A PipelineTask referenced by the Pipeline Result didn’t emit the referenced Task Result. This should be considered a bug in the Task and may fail a PipelineTask in future.

It looks like we want to fail the run if the results are not emitted but referenced?
You case is like the results may or may not emit and it depends on the task's when condition.

I'm ok with @kyubisation's suggestion.

[thomascube]

@Yongxuanzhang apparently I read the following statement differently

This should be considered a bug in the Task and may fail a PipelineTask in the future.

For me this sounds like a case where a task specifies a result but no write to that result file was made in any of the steps. I didn't make the connection to skipped tasks.

Anyway, you asked about my use-case: we have pipelines for building software projects such as a Java Maven application. The pipelines includes tasks for build the jar, testing it, running quality checks (sonar scan) and creating a Docker image from it. Some tasks are optional (e.g. sonar scan or docker build) and can be enabled/disabled by pipeline parameters consumed by when expressions guarding the task execution. These tasks also emit results (if executed) which are mapped into pipeline results. If for example a docker image was built, the image name and tag are visible in the pipeline results and vice versa: if the docker build was disabled by parameter, no such pipeline result exists.

Or a bit shorter: we run pipelines with optional tasks that yield optional results.

I was just surprised to suddenly see pipeline fail although all tasks have executed successfully. For possible solutions I can just copy @kyubisation's suggestions.

[Yongxuanzhang]

Thanks for your input and suggestions! @thomascube @kyubisation
I have a better understanding of the use cases. I will fix it soon!
Will keep you updated. :D

[Yongxuanzhang]

/assign
I believe tektoncd/community#954 is a long term solution. But we should fix this to unblock the use cases