Helm lookup Function Support

[rajivml]

Hello,

Happy new year Guys !!

So, I have this requirement to build the imagePath by reading the dockerRegistryIP value from configMap, so that I need not ask user explicitly where the registry is located.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ template "helm-guestbook.fullname" . }}
    spec:
      containers:
        - name: {{ .Chart.Name }}
          image: {{ printf "%s/%s:%s" **$dockerRegistryIP** .Values.image.repository .Values.image.tag }}

Helm3 has introduced support for this where they introduced a lookup function through which configMap can be read at runtime like this,

{{ (lookup "v1" "ConfigMap" "default" "my-configmap").data.registryURL}}

But the lookup function will return nil when templates are rendered using "helm dryrun" or "helm template" as a result when you parse a field on nil, you will see an exception like this,

"nil pointer evaluating interface {}.registryURL Use --debug flag to render out invalid YAML"

The solution which was proposed on stack overflow is to use "helm template --validate" instead of "helm template"

Can you guys add support for this ?

Right now am populating docker-registry-ip like this, but with this kustomize-plugin approach am loosing the ability to render values.yaml file as an config screen through which user can override certain values i.e. the fix to solve one issue has lead to an other issue

kubectl -n argocd get cm argocd-cm -o yaml
apiVersion: v1
data:
  configManagementPlugins: |
    - name: kustomized-helm
      generate:
        command: [sh, -c]
        args: ["DOCKER_REG_IP=$(kubectl -n registry get svc registry -o jsonpath={.spec.clusterIP}) && sed -i \"s/DOCKER_REGISTRY_IP/$DOCKER_REG_IP/g\" kustomization.yaml | helm template $ARGOCD_APP_NAME --namespace $ARGOCD_APP_NAMESPACE . > all.yaml && kustomize build"]
    - name: kustomized
      generate:
        command: [sh, -c]
        args: ["DOCKER_REG_IP=$(kubectl -n registry get svc registry -o jsonpath={.spec.clusterIP}) && sed -i \"s/DOCKER_REGISTRY_IP/$DOCKER_REG_IP/g\" kustomization.yaml | kustomize build"]

[jessesuen]

Note that even if we allowed configuring Argo CD to append the --validate arg when running the helm template command, the repo-server would still need to be given API server credentials (i.e. mount a service account token) in order to perform the lookup. We would not ever allow kubernetes credentials to repo-server as a default (though you are welcome to modify your deployment in your environment) so there would not be a value in adding a --validate option configuration.

Since you would anyways need a customized repo-server, you can already accomplish this today using a wrapper script around the helm binary which appends the argument in the script (coupled with the service account given to the repo server).

[kvaps]

@jessesuen, I guess this workaround is possible only with in-cluster configuration, and wont work for the external ones.

[jessesuen]

Ah yes, you are right about that unfortunately.

[Gowiem]

@jessesuen just coming across this issue and I'm running into the same. There is duplicate issue here as well: #3640

You mentioned two things to accomplish as a work around for Argo not supporting this:

1. Add a service account for Argo + mount the token -- This is straight forward and would be easy to implement.
2. "using a wrapper script around the helm binary which appends the argument in the script" -- This I don't really get.

Can you expand more on the wrapper script? How would one inject that into a standard Argo deployment?

[dvcanton]

Hi @jessesuen, @Gowiem,

any updates on this?

Thanks in advance, Dave

[Gowiem]

@dvcanton I tried the Argo plugin / wrapper script approach that @jessesuen mentioned after asking about it directly in the Argo Slack. You can find more about that by looking at the plugins documentation.

Unfortunately, that solution seemed overly hacky and pretty esoteric to me and my team. Instead we've now moved towards not using lookup in our charts and copy / pasting certain configuration manually instead. It's not great and I wish Argo would support this, but doesn't seem like there is enough momentum unfortunately.

[randrusiak]

A lot of charts use build-in objects such as Capabilities to provide backward compatibility for old APIs. Capabilities.APIVersions works properly only with --validate flag because without this flag it returns only API versions without available resources.
There is an example in grafana chart: https://github.com/grafana/helm-charts/blob/main/charts/grafana/templates/ingress.yaml#L7

[kvaps]

As about Capabilities, helm template command supports setting capabilities manually, ref #3594

[randrusiak]

@kvaps take a look for an example which I posted.
{{- $newAPI := .Capabilities.APIVersions.Has "networking.k8s.io/v1/Ingress" -}} it returns false because without --validate flag only APIs versions are returned.

[kvaps]

@randrusiak, it works to me:

# helm template . --set ingress.enabled=true --include-crds > /tmp/1.yaml
# helm template . --api-versions networking.k8s.io/v1/Ingress --set ingress.enabled=true --include-crds > /tmp/2.yaml
# diff -u /tmp/1.yaml /tmp/2.yaml
@@ -399,7 +399,7 @@
           emptyDir: {}
 ---
 # Source: grafana/templates/ingress.yaml
-apiVersion: extensions/v1beta1
+apiVersion: networking.k8s.io/v1
 kind: Ingress
 metadata:
   name: RELEASE-NAME-grafana
@@ -417,9 +417,12 @@
         paths:
 
           - path: /
+            pathType: Prefix
             backend:
-              serviceName: RELEASE-NAME-grafana
-              servicePort: 80
+              service:
+                name: RELEASE-NAME-grafana
+                port:
+                  number: 80

my idea was that ArgoCD could provide repo-server list of api-versions from destination Kubernets API. eg:

kubectl api-versions

will return all available apiversions for the cluster.

Not sure if lookup function support can be implemented with the same simplicity, as it already requires direct access to the cluster.

[randrusiak]

@kvaps I understand how it works on the helm level, but I don't know how to pass this additional flag--api-versions networking.k8s.io/v1/Ingress via argo manifest. It's still unclear to me. Could you explain that to me? I'd appreciate your help.

[kvaps]

Actually my note was more likely for contributors than for users :)
They could implement api-versions passing from the argocd-application-controller to the argocd-repo-server via API call.

On current stage, I think you got nothing to do. The only workaround for you is to add serviceAccount to your repo-server and use --validate flag for helm template (you would need to create small shell wrapper script for helm command, or use custom plugin). Unfortunately this is less secure and would work only with the single cluster (the current one).

Another option for you is to hardcode those parameters somewhere, eg. save the output of the following command:

kubectl api-versions | awk '{printf " --api-versions " $1 } END{printf "\n"}' 

And pass it to helm, using any suitable way for you, eg, you can still use wrapper script for helm, something like that:

cat /usr/local/bin/helm
#!/bin/sh
exec /usr/local/bin/helm.bin $HELM_EXTRA_ARGS "$@"

where:

• /usr/local/bin/helm.bin - the original helm binary
• #HELM_EXTRA_ARGS - extra variable for your repo-server

or using custom-plugin