[prometheus-kube-stack] serviceaccount token is not readable for EKS 1.21 - permission denied #1835
Description
Describe the bug a clear and concise description of what the bug is.
The kube-prometheus-stack-admission-create job is failing with error - "error":"open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied"
I think this will be a generic issue with the entire chart. Amazon EKS supports IAM Roles for Service Accounts. It mounts tokens files to /var/run/secrets/eks.amazonaws.com/serviceaccount/token but to read the token we would need to set the securityContext.fsGroup field as per this document
The solution here works for individual deployment by adding the securityContext.fsGroup, however, I am not able to find a way to apply this on the entire chart and its dependency.
What's your helm version?
version.BuildInfo{Version:"v3.8.0", GitCommit:"d14138609b01886f544b2025f5000351c9eb092e", GitTreeState:"clean", GoVersion:"go1.17.5"}
What's your kubectl version?
v1.22.0
Which chart?
prometheus-operator
What's the chart version?
33.0.0
What happened?
The kube-prometheus-stack-admission-create job is failing with the below error
W0228 13:28:10.878848 1 client_config.go:615] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
W0228 13:28:10.878925 1 client_config.go:620] error creating inClusterConfig, falling back to default config: open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied
{"error":"open /var/run/secrets/kubernetes.io/serviceaccount/token: permission denied","level":"fatal","msg":"error building kubernetes config","source":"cmd/root.go:89","time":"2022-02-28T13:28:10Z"}
The solution here works for individual deployment by adding the securityContext.fsGroup, however, I am not able to find a way to apply this on the entire chart and its dependency.
What you expected to happen?
Chart to be deployed and with no error.
How to reproduce it?
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm upgrade --install prom -n mon prometheus-community/kube-prometheus-stack --version 33.0.0
Enter the changed values of values.yaml?
None
Enter the command that you execute and failing/misfunctioning.
helm upgrade --install prom -n mon prometheus-community/kube-prometheus-stack --version 33.0.0
Anything else we need to know?
No response