feat: add support for authenticated git clone

Added support for authenticated git cloning using `gitToken` and `gitTokenKey`
parameters. Updated documentation to reflect the new parameters and their
usage. Modified resolver functions to handle authentication when cloning
repositories. Added tests to verify the functionality of authenticated git
cloning.

The differences between the two modes are:

- The `git clone` method support anonymous cloning and authenticated cloning.
- Depending of the Git provider `git clone` has a lower rate limit (if none)
  than the authenticated API.
- The authenticated API supports private repositories and fetches only the file
  at the specified path rather than doing a full clone.

Signed-off-by: Chmouel Boudjnah <[email protected]>

Author: chmouel

docs/git-resolver.md

@@ -13,17 +13,19 @@ This Resolver responds to type `git`.
 
 ## Parameters
 
-| Param Name   | Description                                                                                                                                                                | Example Value                                               |
-|--------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------|
-| `url`        | URL of the repo to fetch and clone anonymously. Either `url`, or `repo` (with `org`) must be specified, but not both.                                                      | `https://github.com/tektoncd/catalog.git`                   |
-| `repo`       | The repository to find the resource in. Either `url`, or `repo` (with `org`) must be specified, but not both.                                                              | `pipeline`, `test-infra`                                    |
-| `org`        | The organization to find the repository in. Default can be set in [configuration](#configuration).                                                                         | `tektoncd`, `kubernetes`                                    |
-| `token`      | An optional secret name in the `PipelineRun` namespace to fetch the token from. Defaults to empty, meaning it will try to use the configuration from the global configmap. | `secret-name`, (empty)                                      |
-| `tokenKey`   | An optional key in the token secret name in the `PipelineRun` namespace to fetch the token from. Defaults to `token`.                                                      | `token`                                                     |
-| `revision`   | Git revision to checkout a file from. This can be commit SHA, branch or tag.                                                                                               | `aeb957601cf41c012be462827053a21a420befca` `main` `v0.38.2` |
-| `pathInRepo` | Where to find the file in the repo.                                                                                                                                        | `task/golang-build/0.3/golang-build.yaml`                   |
-| `serverURL`  | An optional server URL (that includes the https:// prefix) to connect for API operations                                                                                   | `https:/github.mycompany.com`                               |
-| `scmType`    | An optional SCM type to use for API operations                                                                                                                             | `github`, `gitlab`, `gitea`                                 |
+| Param Name    | Description                                                                                                                                                                | Example Value                                               |
+|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------|
+| `url`         | URL of the repo to fetch and clone anonymously. Either `url`, or `repo` (with `org`) must be specified, but not both.                                                      | `https://github.com/tektoncd/catalog.git`                   |
+| `repo`        | The repository to find the resource in. Either `url`, or `repo` (with `org`) must be specified, but not both.                                                              | `pipeline`, `test-infra`                                    |
+| `org`         | The organization to find the repository in. Default can be set in [configuration](#configuration).                                                                         | `tektoncd`, `kubernetes`                                    |
+| `token`       | An optional secret name in the `PipelineRun` namespace to fetch the token from. Defaults to empty, meaning it will try to use the configuration from the global configmap. | `secret-name`, (empty)                                      |
+| `tokenKey`    | An optional key in the token secret name in the `PipelineRun` namespace to fetch the token from. Defaults to `token`.                                                      | `token`                                                     |
+| `gitToken`       | An optional secret name in the `PipelineRun` namespace to fetch the token from when doing opration with the `git clone`. When empty it will use anonymous cloning. | `secret-gitauth-token` |
+| `gitTokenKey` | An optional key in the token secret name in the `PipelineRun` namespace to fetch the token from when using the `git clone`. Defaults to `token`.                                                      | `token`                                                     |
+| `revision`    | Git revision to checkout a file from. This can be commit SHA, branch or tag.                                                                                               | `aeb957601cf41c012be462827053a21a420befca` `main` `v0.38.2` |
+| `pathInRepo`  | Where to find the file in the repo.                                                                                                                                        | `task/golang-build/0.3/golang-build.yaml`                   |
+| `serverURL`   | An optional server URL (that includes the https:// prefix) to connect for API operations                                                                                   | `https:/github.mycompany.com`                               |
+| `scmType`     | An optional SCM type to use for API operations                                                                                                                             | `github`, `gitlab`, `gitea`                                 |
 
 ## Requirements
 
@@ -55,11 +57,25 @@ for the name, namespace and defaults that the resolver ships with.
 
 ## Usage
 
-The `git` resolver has two modes: cloning a repository anonymously, or fetching individual files via an SCM provider's API using an API token.
+The `git` resolver has two modes: cloning a repository with `git clone` (with
+the `go-git` library), or fetching individual files via an SCM provider's API
+using an API token.
 
-### Anonymous Cloning
+The differences between the two modes are:
 
-Anonymous cloning is supported only for public repositories. This mode clones the full git repo.
+- The `git clone` method support anonymous cloning and authenticated cloning.
+- When it comes to throughput, `git clone` is a more efficient option because
+  it is not subject to the same rate limits as API calls. This is a key
+  difference between the two modes: `git clone` supports both anonymous and
+  authenticated cloning, and depending on the Git provider, it often has a
+  higher rate limit compared to authenticated API calls.
+- The `git clone` method is inefficient for larger repositories, as it clones the
+  entire repository in memory, whereas the authenticated API fetches only the
+  files at the specified path.
+
+### Git Clone with git clone
+
+Git clone with `git clone` is supported for anonymous and authenticated cloning.
 
 #### Task Resolution
 
@@ -78,6 +94,11 @@ spec:
       value: main
     - name: pathInRepo
       value: task/git-clone/0.6/git-clone.yaml
+    # Uncomment the following lines to use a secret with a token
+    # - name: gitToken
+    #   value: "secret-with-token"
+    # - name: gitTokenKey (optional, defaults to "token")
+    #   value: "token"
 ```
 
 #### Pipeline resolution
@@ -97,6 +118,11 @@ spec:
       value: main
     - name: pathInRepo
       value: pipeline/simple/0.1/simple.yaml
+    # Uncomment the following lines to use a secret with a token
+    # - name: gitToken
+    #   value: "secret-with-token"
+    # - name: gitTokenKey (optional, defaults to "token")
+    #   value: "token"
   params:
   - name: name
     value: Ranni
@@ -108,11 +134,12 @@ The authenticated API supports private repositories, and fetches only the file a
 
 When using the authenticated API, [providers with implementations in `go-scm`](https://github.com/jenkins-x/go-scm/tree/main/scm/driver) can be used.
 Note that not all `go-scm` implementations have been tested with the `git` resolver, but it is known to work with:
-  * github.com and GitHub Enterprise
-  * gitlab.com and self-hosted Gitlab
-  * Gitea
-  * BitBucket Server
-  * BitBucket Cloud
+
+- github.com and GitHub Enterprise
+- gitlab.com and self-hosted Gitlab
+- Gitea
+- BitBucket Server
+- BitBucket Cloud
 
 #### Task Resolution
 
@@ -193,11 +220,11 @@ spec:
 
 ### Specifying Configuration for Multiple Git Providers
 
-It is possible to specify configurations for multiple providers and even multiple configurations for same provider to use in 
+It is possible to specify configurations for multiple providers and even multiple configurations for same provider to use in
 different tekton resources. Firstly, details need to be added in configmap with the unique identifier key prefix.
-To use them in tekton resources, pass the unique key mentioned in configmap as an extra param to resolver with key 
-`configKey` and value will be the unique key. If no `configKey` param is passed, `default` will be used. Default 
-configuration to be used for git resolver can be specified in configmap by either mentioning no unique identifier or 
+To use them in tekton resources, pass the unique key mentioned in configmap as an extra param to resolver with key
+`configKey` and value will be the unique key. If no `configKey` param is passed, `default` will be used. Default
+configuration to be used for git resolver can be specified in configmap by either mentioning no unique identifier or
 using identifier `default`
 
 **Note**: `configKey` should not contain `.` while specifying configurations in configmap
@@ -253,7 +280,7 @@ data:
 
 #### Task Resolution
 
-A specific configurations from the configMap can be selected by passing the parameter `configKey` with the value 
+A specific configurations from the configMap can be selected by passing the parameter `configKey` with the value
 matching one of the configuration keys used in the configMap.
 
 ```yaml
@@ -304,17 +331,21 @@ spec:
 ```
 
 ## `ResolutionRequest` Status
+
 `ResolutionRequest.Status.RefSource` field captures the source where the remote resource came from. It includes the 3 subfields: `url`, `digest` and `entrypoint`.
+
 - `url`
   - If users choose to use anonymous cloning, the url is just user-provided value for the `url` param in the [SPDX download format](https://spdx.github.io/spdx-spec/package-information/#77-package-download-location-field).
   - If scm api is used, it would be the clone URL of the repo fetched from scm repository service in the [SPDX download format](https://spdx.github.io/spdx-spec/package-information/#77-package-download-location-field).
 - `digest`
-  - The algorithm name is fixed "sha1", but subject to be changed to "sha256" once Git eventually uses SHA256 at some point later. See https://git-scm.com/docs/hash-function-transition for more details.
+  - The algorithm name is fixed "sha1", but subject to be changed to "sha256" once Git eventually uses SHA256 at some point later. See <https://git-scm.com/docs/hash-function-transition> for more details.
   - The value is the actual commit sha at the moment of resolving the resource even if a user provides a tag/branch name for the param `revision`.
 - `entrypoint`: the user-provided value for the `path` param.
 
 Example:
+
 - Pipeline Resolution
+
 ```yaml
 apiVersion: tekton.dev/v1beta1
 kind: PipelineRun
@@ -333,6 +364,7 @@ spec:
 ```
 
 - `ResolutionRequest`
+
 ```yaml
 apiVersion: resolution.tekton.dev/v1alpha1
 kind: ResolutionRequest

pkg/remoteresolution/resolver/git/resolver.go

@@ -119,11 +119,19 @@ func (r *Resolver) Resolve(ctx context.Context, req *v1beta1.ResolutionRequestSp
 			return nil, err
 		}
 
+		g := &git.GitResolver{
+			KubeClient: r.kubeClient,
+			Logger:     r.logger,
+			Cache:      r.cache,
+			TTL:        r.ttl,
+			Params:     params,
+		}
+
 		if params[git.UrlParam] != "" {
-			return git.ResolveAnonymousGit(ctx, params)
+			return g.ResolveGitClone(ctx)
 		}
 
-		return git.ResolveAPIGit(ctx, params, r.kubeClient, r.logger, r.cache, r.ttl, r.clientFunc)
+		return g.ResolveAPIGit(ctx, r.clientFunc)
 	}
 	// Remove this error once resolution of url has been implemented.
 	return nil, errors.New("the Resolve method has not been implemented.")

pkg/resolution/resolver/git/config.go

@@ -62,6 +62,7 @@ type ScmConfig struct {
 	Org                string `json:"default-org"`
 	ServerURL          string `json:"server-url"`
 	SCMType            string `json:"scm-type"`
+	GitToken           string `json:"git-token"`
 	APISecretName      string `json:"api-token-secret-name"`
 	APISecretKey       string `json:"api-token-secret-key"`
 	APISecretNamespace string `json:"api-token-secret-namespace"`

pkg/resolution/resolver/git/params.go

@@ -33,6 +33,10 @@ const (
 	TokenParam string = "token"
 	// TokenKeyParam is an optional reference to a key in the TokenParam secret for SCM API authentication
 	TokenKeyParam string = "tokenKey"
+	// GitTokenParam is an optional reference to a secret name when using go-git for git authentication
+	GitTokenParam string = "gitToken"
+	// GitTokenParam is an optional reference to a secret name when using go-git for git authentication
+	GitTokenKeyParam string = "gitTokenKey"
 	// DefaultTokenKeyParam is the default key in the TokenParam secret for SCM API authentication
 	DefaultTokenKeyParam string = "token"
 	// scmTypeParam is an optional string overriding the scm-type configuration (ie: github, gitea, gitlab etc..)