Add support for implicit param mapping

[seansund]

Feature request

Currently, mapping parameters to tasks is a tedious process. The controller is also very unforgiving if there is an error in the mapping and will often fail with error messages that don't reveal the real issue.

There seem to be a couple of common scenarios for how parameters are mapped (pipeline -> task, task -> following task, etc). The proposed solution would be to apply logic in the following order (all assuming the scenario where a param value is NOT explicitly provided):

1. If the previous task (ideally, any of the previous tasks) has a result whose name exactly matches the input param of the task, use the output from the previous task
2. If the pipeline has an input param whose name exactly matches the input param of the task, use the pipeline input
3. If there are no matches to the input param name then give an error

This "autowiring" feature could be controlled via an annotation on the pipeline so that the pipeline must "opt in" to this behavior to avoid accidental

Use case

Here's a pipeline we are using that contains a number of steps that have many common params:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: ibm-nodejs
spec:
  params:
    - name: git-url
      description: The url for the git repository
    - name: git-revision
      description: The git revision (branch, tag, or sha) that should be built
      default: master
    - name: image-url
      description: The dev image (use image-release for released image)
      default: ""
    - name: image-release
      description: The url of final released image
      default: ""
    - name: scan-image
      description: Flag indicating that a Vulnerability Advisor scan should be performed
      default: "false"
  tasks:
    - name: setup
      taskRef:
        name: ibm-setup
      params:
        - name: git-url
          value: $(params.git-url)
        - name: git-revision
          value: $(params.git-revision)
        - name: image-url
          value: $(params.image-url)
        - name: image-release
          value: $(params.image-release)
        - name: scan-image
          value: $(params.scan-image)
    - name: test
      taskRef:
        name: ibm-nodejs-test
      runAfter:
        - setup
      params:
        - name: git-url
          value: "$(tasks.setup.results.git-url)"
        - name: git-revision
          value: "$(tasks.setup.results.git-revision)"
        - name: source-dir
          value: "$(tasks.setup.results.source-dir)"
        - name: js-image
          value: "$(tasks.setup.results.js-image)"
    - name: build
      taskRef:
        name: ibm-build-tag-push
      runAfter:
        - test
      params:
        - name: git-url
          value: "$(tasks.setup.results.git-url)"
        - name: git-revision
          value: "$(tasks.setup.results.git-revision)"
        - name: source-dir
          value: "$(tasks.setup.results.source-dir)"
        - name: image-server
          value: "$(tasks.setup.results.image-server)"
        - name: image-namespace
          value: "$(tasks.setup.results.image-namespace)"
        - name: image-repository
          value: "$(tasks.setup.results.image-repository)"
        - name: image-tag
          value: "$(tasks.setup.results.image-tag)"
    - name: deploy
      taskRef:
        name: ibm-deploy
      runAfter:
        - build
      params:
        - name: git-url
          value: "$(tasks.setup.results.git-url)"
        - name: git-revision
          value: "$(tasks.setup.results.git-revision)"
        - name: source-dir
          value: "$(tasks.setup.results.source-dir)"
        - name: image-server
          value: "$(tasks.setup.results.image-server)"
        - name: image-namespace
          value: "$(tasks.setup.results.image-namespace)"
        - name: image-repository
          value: "$(tasks.setup.results.image-repository)"
        - name: image-tag
          value: "$(tasks.setup.results.image-tag)"
        - name: app-namespace
          value: "$(tasks.setup.results.app-namespace)"
        - name: app-name
          value: "$(tasks.setup.results.app-name)"
        - name: deploy-ingress-type
          value: "$(tasks.setup.results.deploy-ingress-type)"
        - name: tools-image
          value: "$(tasks.setup.results.tools-image)"
    - name: health
      taskRef:
        name: ibm-health-check
      runAfter:
        - deploy
      params:
        - name: app-namespace
          value: "$(tasks.setup.results.app-namespace)"
        - name: app-name
          value: "$(tasks.setup.results.app-name)"
        - name: deploy-ingress-type
          value: "$(tasks.setup.results.deploy-ingress-type)"
        - name: health-protocol
          value: "$(tasks.setup.results.health-protocol)"
        - name: health-endpoint
          value: "$(tasks.setup.results.health-endpoint)"
        - name: tools-image
          value: "$(tasks.setup.results.tools-image)"
    - name: tag-release
      taskRef:
        name: ibm-tag-release
      runAfter:
        - health
      params:
        - name: git-url
          value: "$(tasks.setup.results.git-url)"
        - name: git-revision
          value: "$(tasks.setup.results.git-revision)"
        - name: source-dir
          value: "$(tasks.setup.results.source-dir)"
        - name: js-image
          value: "$(tasks.setup.results.js-image)"
    - name: img-release
      taskRef:
        name: ibm-img-release
      runAfter:
        - tag-release
      params:
        - name: image-from
          value: "$(tasks.setup.results.image-url)"
        - name: image-to
          value: "$(tasks.setup.results.image-release):$(tasks.tag-release.results.tag)"
    - name: scan
      taskRef:
        name: ibm-img-scan-ibm
      runAfter:
        - img-release
      params:
        - name: image-url
          value: $(tasks.img-release.results.image-url)
        - name: scan-image
          value: $(params.scan-image)
    - name: helm-release
      taskRef:
        name: ibm-helm-release
      runAfter:
        - scan
      params:
        - name: git-url
          value: "$(tasks.setup.results.git-url)"
        - name: git-revision
          value: "$(tasks.setup.results.git-revision)"
        - name: source-dir
          value: "$(tasks.setup.results.source-dir)"
        - name: image-url
          value: "$(tasks.img-release.results.image-url)"
        - name: app-name
          value: "$(tasks.setup.results.app-name)"
        - name: deploy-ingress-type
          value: "$(tasks.setup.results.deploy-ingress-type)"
        - name: tools-image
          value: "$(tasks.setup.results.tools-image)"
    - name: gitops
      taskRef:
        name: ibm-gitops
      runAfter:
        - helm-release
      params:
        - name: app-name
          value: "$(tasks.setup.results.app-name)"
        - name: version
          value: "$(tasks.tag-release.results.tag)"
        - name: helm-url
          value: $(tasks.helm-release.results.helm-url)
        - name: tools-image
          value: "$(tasks.setup.results.tools-image)"

With the annotation the pipeline would be reduced to something like the following. (The annotation has been added and most all of the params blocks in the tasks section have been removed because they would be auto-wired.)

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: ibm-nodejs
  annotation:
    tekton/auto-wire-params: true
spec:
  params:
    - name: git-url
      description: The url for the git repository
    - name: git-revision
      description: The git revision (branch, tag, or sha) that should be built
      default: master
    - name: image-url
      description: The dev image (use image-release for released image)
      default: ""
    - name: image-release
      description: The url of final released image
      default: ""
    - name: scan-image
      description: Flag indicating that a Vulnerability Advisor scan should be performed
      default: "false"
  tasks:
    - name: setup
      taskRef:
        name: ibm-setup
    - name: test
      taskRef:
        name: ibm-nodejs-test
      runAfter:
        - setup
    - name: build
      taskRef:
        name: ibm-build-tag-push
      runAfter:
        - test
    - name: deploy
      taskRef:
        name: ibm-deploy
      runAfter:
        - build
    - name: health
      taskRef:
        name: ibm-health-check
      runAfter:
        - deploy
    - name: tag-release
      taskRef:
        name: ibm-tag-release
      runAfter:
        - health
    - name: img-release
      taskRef:
        name: ibm-img-release
      runAfter:
        - tag-release
      params:
        - name: image-from
          value: "$(tasks.setup.results.image-url)"
        - name: image-to
          value: "$(tasks.setup.results.image-release):$(tasks.tag-release.results.tag)"
    - name: scan
      taskRef:
        name: ibm-img-scan-ibm
      runAfter:
        - img-release
    - name: helm-release
      taskRef:
        name: ibm-helm-release
      runAfter:
        - scan
    - name: gitops
      taskRef:
        name: ibm-gitops
      runAfter:
        - helm-release
      params:
        - name: version
          value: "$(tasks.tag-release.results.tag)"

[vdemeester]

This "autowiring" feature could be controlled via an annotation on the pipeline so that the pipeline must "opt in" to this behavior to avoid accidental

I do like the idea of having this behavior "opt in" 👼

[afrittoli]

FY: #1484

[dlorenc]

I like it!

[chmouel]

Woo really cool idea 👍

[bobcatfish]

Could we include an example of what the automatically wired version of above would look like?

[Peaorl]

I've discussed this feature with @sbwsg and would like to get started on this. My first step will be to write up a TEP.

@bobcatfish, I'll go ahead with the example @afrittoli provided #1484.

/assign @Peaorl

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[vdemeester]

/remove-lifecycle stale

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.