bug: Strange behavior of minio in trace -v mode

[Viktor3434]

Strange behavior of minio in trace -v mode

Expected Behavior

absence of errors

Current Behavior

frequent 403 Forbidden error in trace
(mc admin trace -v myminio)

Possible Solution

I don't know.

Steps to Reproduce (for bugs)

1. deploy namespace
2. deploy operator
3. deploy tenant.

Context

I'm just deploying an operator + tenant instance, and I wouldn't want it to have a negative impact on the clusterer in the future

Regression

Your Environment

• Version used (minio-operator):
  version: 6.0.1
  appVersion: v6.0.1
• Environment name and version (e.g. kubernetes v1.17.2): k8s v1.21.5
• Server type and version: managed k8s
• Operating System and version (uname -a):
• Link to your deployment file:

secrets:
  name:
  accessKey: 
  secretKey:

tenant:
  name: minio-tenant

  image:
    repository: minio/minio
    tag: RELEASE.2024-07-13T01-46-15Z
    pullPolicy: IfNotPresent

  configuration:
    name: minio-tenant
  configSecret:
    name: minio-tenant
    accessKey:
    secretKey:
    existingSecret: true

  pools:
    - servers: 4
      name: pool-0
      volumesPerServer: 1
      size: 30Gi
      storageClassName: default
  mountPath: /export
  subPath: /data

  buckets:
    - name: my-minio-bucket

  env:
    - name: MINIO_IDENTITY_OPENID_CONFIG_URL
      value: https://my.keycloak.org
    - name: MINIO_BROWSER_REDIRECT_URL
      value: https:/minio-api-test.my.org
    - name: MINIO_IDENTITY_OPENID_CLIENT_ID
      value: minio
    - name: MINIO_IDENTITY_OPENID_DISPLAY_NAME
      value: KEYCLOAK
    - name: MINIO_IDENTITY_OPENID_CLAIM_USERINFO
      value: "on"
    - name: MINIO_IDENTITY_OPENID_SCOPES
      value: profile

ingress:
  api:
    enabled: true
    ingressClassName: ""
    labels: { }
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/proxy-body-size: "0"
      nginx.ingress.kubernetes.io/backend-protocol: https
    tls:
      - hosts:
          - minio-api-test.my.org
        secretName: wild-my-org
    host: minio-api-test.my.org
    path: /
    pathType: Prefix
  console:
    enabled: true
    ingressClassName: ""
    labels: { }
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/proxy-body-size: "0"
      nginx.ingress.kubernetes.io/backend-protocol: https
    tls:
      - hosts:
          - dp-minio-test.my.org
        secretName: wild-my-org  
    host: dp-minio-test.my.org
    path: /
    pathType: Prefix

• Trace errors (repeated constantly and for all servers in the pool)

minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 [REQUEST s3.ListBuckets] [2024-08-07T12:32:36.050] [Client IP: [::1]]
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 GET /
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Proto: HTTP/1.1
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Host: localhost:9000
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Accept-Encoding: gzip
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Content-Length: 0
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 User-Agent: Go-http-client/1.1
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 [RESPONSE] [2024-08-07T12:32:36.050] [ Duration 173µs TTFB 165.923µs ↑ 47 B  ↓ 254 B ]
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 403 Forbidden
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 X-Xss-Protection: 1; mode=block
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Accept-Ranges: bytes
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Content-Length: 254
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Content-Type: application/xml
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Server: MinIO
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Vary: Origin,Accept-Encoding
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 X-Amz-Id-2: 0f50a5b6e25b931632da5cr086d72128c3eeb73ee38d7ce87432ec187d9673dd
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 X-Amz-Request-Id: 17E972410656F2F4
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 X-Content-Type-Options: nosniff
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Resource>/</Resource><RequestId>17E972410656F2F4</RequestId><HostId>0f50a5b6e23b931632da5cf086d72128c3eeb73ee38d7ce87432ec187d9673dd</HostId></Error>
minio-tenant-pool-0-2.minio-tenant-hl.minio-tenant.svc.cluster.local:9000
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 [REQUEST s3.ListBuckets] [2024-08-07T12:32:36.104] [Client IP: [::1]]
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 GET /
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Proto: HTTP/1.1
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Host: localhost:9000
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 User-Agent: Go-http-client/1.1
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Accept-Encoding: gzip
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Content-Length: 0
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 [RESPONSE] [2024-08-07T12:32:36.104] [ Duration 126µs TTFB 120.116µs ↑ 47 B  ↓ 254 B ]
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 403 Forbidden
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 X-Amz-Request-Id: 17E972410987B9E5
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 X-Content-Type-Options: nosniff
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 X-Xss-Protection: 1; mode=block
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Accept-Ranges: bytes
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Server: MinIO
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Vary: Origin,Accept-Encoding
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 X-Amz-Id-2: 8f95f7c903a18b8d55ffe798f8d9c4e35be74957c03e41b411f4c18a0de797b0
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Content-Length: 254
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 Content-Type: application/xml
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000 <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied.</Message><Resource>/</Resource><RequestId>17E972410987B9E5</RequestId><HostId>8f95f7c903a18b8d54ffe798f8d9c4e35be74957c03e41b411f4c18a0ae797b0</HostId></Error>
minio-tenant-pool-0-0.minio-tenant-hl.minio-tenant.svc.cluster.local:9000

[ramondeklein]

Thanks for your detailed report. This is the health-check that is hitting the endpoint and shows up as ListBuckets (GET / is s3:ListBuckets). Fixed in #2257.