[BUG] Constant recreation of DNS records (CloudFlare) if ingress-nginx has multiple hosts

[kinseii]

What happened:

Constant recreation of DNS records (CloudFlare) if ingress-nginx has multiple hosts:

time="2025-10-02T19:37:14Z" level=info msg="Changing record." action=DELETE record=llm2.example.com ttl=300 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:37:15Z" level=info msg="Changing record." action=DELETE record=llm1.example.com ttl=300 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:37:15Z" level=info msg="Changing record." action=DELETE record=gptoss120b-qwen3emb8b.example.com ttl=300 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:37:16Z" level=info msg="Changing record." action=DELETE record=a-llm2.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:37:16Z" level=info msg="Changing record." action=DELETE record=a-llm1.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:37:17Z" level=info msg="Changing record." action=DELETE record=a-gptoss120b-qwen3emb8b.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:38:15Z" level=info msg="Changing record." action=CREATE record=gptoss120b-qwen3emb8b.example.com ttl=1 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:38:16Z" level=info msg="Changing record." action=CREATE record=llm2.example.com ttl=1 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:38:16Z" level=info msg="Changing record." action=CREATE record=llm1.example.com ttl=1 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:38:17Z" level=info msg="Changing record." action=CREATE record=a-gptoss120b-qwen3emb8b.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:38:17Z" level=info msg="Changing record." action=CREATE record=a-llm2.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:38:17Z" level=info msg="Changing record." action=CREATE record=a-llm1.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:39:15Z" level=info msg="All records are already up to date"
time="2025-10-02T19:40:15Z" level=info msg="All records are already up to date"
time="2025-10-02T19:41:16Z" level=info msg="All records are already up to date"
time="2025-10-02T19:42:17Z" level=info msg="Changing record." action=DELETE record=llm2.example.com ttl=1 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:42:18Z" level=info msg="Changing record." action=DELETE record=gptoss120b-qwen3emb8b.example.com ttl=1 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:42:18Z" level=info msg="Changing record." action=DELETE record=llm1.example.com ttl=1 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:42:19Z" level=info msg="Changing record." action=DELETE record=a-llm2.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:42:19Z" level=info msg="Changing record." action=DELETE record=a-gptoss120b-qwen3emb8b.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:42:20Z" level=info msg="Changing record." action=DELETE record=a-llm1.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:43:18Z" level=info msg="Records cache provider: refreshing records list cache"
time="2025-10-02T19:43:19Z" level=info msg="Changing record." action=CREATE record=llm1.example.com ttl=1 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:43:20Z" level=info msg="Changing record." action=CREATE record=gptoss120b-qwen3emb8b.example.com ttl=1 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:43:20Z" level=info msg="Changing record." action=CREATE record=llm2.example.com ttl=1 type=A zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:43:20Z" level=info msg="Changing record." action=CREATE record=a-llm1.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:43:21Z" level=info msg="Changing record." action=CREATE record=a-gptoss120b-qwen3emb8b.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8
time="2025-10-02T19:43:21Z" level=info msg="Changing record." action=CREATE record=a-llm2.example.com ttl=1 type=TXT zone=c1f6c1613ec42a2ce8e16cc5c3b8f9c8

Ingress-nginx manifest:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-body-size: 200m
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/use-regex: "true"
  name: gptoss120b-qwen3emb8b
  namespace: models
spec:
  ingressClassName: nginx
  rules:
  - host: gptoss120b-qwen3emb8b.example.com
    http:
      paths:
      - backend:
          service:
            name: gptoss120b-qwen3emb8b-predictor
            port:
              number: 80
        path: /gptoss120b(/|$)(.*)
        pathType: ImplementationSpecific
      - backend:
          service:
            name: gptoss120b-qwen3emb8b-predictor
            port:
              number: 81
        path: /qwen3emb8b(/|$)(.*)
        pathType: ImplementationSpecific
  - host: llm1.example.com
    http:
      paths:
      - backend:
          service:
            name: gptoss120b-qwen3emb8b-predictor
            port:
              number: 80
        path: /
        pathType: Prefix
  - host: llm2.example.com
    http:
      paths:
      - backend:
          service:
            name: gptoss120b-qwen3emb8b-predictor
            port:
              number: 81
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - gptoss120b-qwen3emb8b.example.com
    - llm1.example.com
    - llm2.example.com
    secretName: gptoss120b-qwen3emb8b-ingress-tls

Ingress-nginx args (--update-status-on-shutdown=false):

    Args:
      /nginx-ingress-controller
      --default-backend-service=$(POD_NAMESPACE)/ingress-nginx-defaultbackend
      --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
      --election-id=ingress-nginx-leader
      --controller-class=k8s.io/ingress-nginx
      --ingress-class=nginx
      --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
      --validating-webhook=:8443
      --validating-webhook-certificate=/usr/local/certificates/cert
      --validating-webhook-key=/usr/local/certificates/key
      --enable-ssl-passthrough
      --update-status-on-shutdown=false

If I create a separate ingress for each host, there are no problems:

time="2025-10-02T21:09:11Z" level=info msg="All records are already up to date"
time="2025-10-02T21:10:12Z" level=info msg="All records are already up to date"
time="2025-10-02T21:11:13Z" level=info msg="All records are already up to date"
time="2025-10-02T21:12:13Z" level=info msg="All records are already up to date"
time="2025-10-02T21:13:14Z" level=info msg="All records are already up to date"
time="2025-10-02T21:14:14Z" level=info msg="Records cache provider: refreshing records list cache"
time="2025-10-02T21:14:15Z" level=info msg="All records are already up to date"
time="2025-10-02T21:15:15Z" level=info msg="All records are already up to date"
time="2025-10-02T21:16:15Z" level=info msg="All records are already up to date"
time="2025-10-02T21:17:16Z" level=info msg="All records are already up to date"
time="2025-10-02T21:18:17Z" level=info msg="All records are already up to date"
time="2025-10-02T21:19:17Z" level=info msg="All records are already up to date"
time="2025-10-02T21:20:17Z" level=info msg="All records are already up to date"
time="2025-10-02T21:21:17Z" level=info msg="All records are already up to date"
time="2025-10-02T21:22:18Z" level=info msg="All records are already up to date"
time="2025-10-02T21:23:19Z" level=info msg="All records are already up to date"
time="2025-10-02T21:24:20Z" level=info msg="All records are already up to date"
time="2025-10-02T21:25:20Z" level=info msg="All records are already up to date"
time="2025-10-02T21:26:20Z" level=info msg="All records are already up to date"
time="2025-10-02T21:27:21Z" level=info msg="All records are already up to date"
time="2025-10-02T21:28:21Z" level=info msg="All records are already up to date"
time="2025-10-02T21:29:21Z" level=info msg="Records cache provider: refreshing records list cache"
time="2025-10-02T21:29:22Z" level=info msg="All records are already up to date"
time="2025-10-02T21:30:22Z" level=info msg="All records are already up to date"
time="2025-10-02T21:31:23Z" level=info msg="All records are already up to date"
time="2025-10-02T21:32:24Z" level=info msg="All records are already up to date"
time="2025-10-02T21:33:25Z" level=info msg="All records are already up to date"
time="2025-10-02T21:34:26Z" level=info msg="All records are already up to date"
time="2025-10-02T21:35:26Z" level=info msg="All records are already up to date"
time="2025-10-02T21:36:27Z" level=info msg="All records are already up to date"
time="2025-10-02T21:37:28Z" level=info msg="All records are already up to date"
time="2025-10-02T21:38:29Z" level=info msg="All records are already up to date"
time="2025-10-02T21:39:29Z" level=info msg="All records are already up to date"
time="2025-10-02T21:40:30Z" level=info msg="All records are already up to date"
time="2025-10-02T21:41:31Z" level=info msg="All records are already up to date"
time="2025-10-02T21:42:31Z" level=info msg="All records are already up to date"
time="2025-10-02T21:43:31Z" level=info msg="All records are already up to date"
time="2025-10-02T21:44:32Z" level=info msg="Records cache provider: refreshing records list cache"
time="2025-10-02T21:44:33Z" level=info msg="All records are already up to date"

What you expected to happen:
Records should not be recreated.

Environment:

• External-DNS version (use external-dns --version): v0.19.0
• DNS provider: CloudFlare
• Others: Kubernetes v1.32

[ivankatliarchuk]

How about version 0.18 is behaviour same?

[kinseii]

How about version 0.18 is behaviour same?

Yes, originally the problem was with v0.18.0 and then I did an upgrade but nothing changed.

[ritik872000]

I can confirm, I have the same issue as well!

in the release notes of 0.19.0 it says the issue has been fixed, but thats not the case.

[u-kai]

I couldn’t reproduce the issue with the following arguments:

--source=ingress \
--provider=cloudflare \
--policy=sync \
--registry=txt \
--log-level=debug \
--interval=10s

Could you share what arguments you’re using to run it?
Also, please check whether multiple ExternalDNS instances are running and if they’re using the same txt-owner-id.

[kinseii]

I couldn’t reproduce the issue with the following arguments:

--source=ingress \
--provider=cloudflare \
--policy=sync \
--registry=txt \
--log-level=debug \
--interval=10s

Could you share what arguments you’re using to run it? Also, please check whether multiple ExternalDNS instances are running and if they’re using the same txt-owner-id.

Hi, thanx for your help! Here External DNS deployment args:

    Args:
      --metrics-address=:7979
      --log-level=info
      --log-format=text
      --domain-filter=example1.com
      --domain-filter=example2.com
      --policy=sync
      --provider=cloudflare
      --registry=txt
      --interval=1m
      --txt-owner-id=qwerty-xxx-xxx-xxx-1
      --source=service
      --source=ingress
      --cloudflare-dns-records-per-page=100
      --provider-cache-time=15m
      --txt-cache-interval=15m
      --txt-wildcard-replacement=wildcard

We use txt-owner-id, that's the first thing I checked.

[kinseii]

I couldn’t reproduce the issue with the following arguments

@u-kai I will double-check on another cluster.

[kinseii]

@u-kai I double-checked on another cluster running version v0.18.0, and everything is OK. This is likely due to Istio and Ingress-nginx running simultaneously. When I have time, I'll make sure to write back here. In the meantime, you can go ahead and close the issue.

[kinseii]

Perhaps it is still worth waiting for @ritik872000 response.

[ritik872000]

Perhaps it is still worth waiting for @ritik872000 response.

hey, this are my config
- --source=service - --source=ingress - --zone-id-filter=<zone id> - --provider=cloudflare - --annotation-filter=external-dns.alpha.kubernetes.io/hostname - --cloudflare-record-comment="provisioned by external-dns"
and I annotate the ing but it keeps on creating the record

I got this image from one of the maintainer
image: gcr.io/k8s-staging-external-dns/external-dns:v20250919-v0.19.0-30-g8851544c he mentioned the images points to master after using this the issue was not there but it started happening again after couple of days

[u-kai]

Thanks a lot for all the details!

@kinseii

This is likely due to Istio and Ingress-nginx running simultaneously.

In your environment, when Istio and Ingress-nginx are running simultaneously, does External-DNS encounter this issue, or does having both running together actually prevent the problem?

@ritik872000

I got this image from one of the maintainers
image: gcr.io/k8s-staging-external-dns/external-dns:v20250919-v0.19.0-30-g8851544c
he mentioned the image points to master; after using this, the issue was gone, but it started happening again after a couple of days.

That’s strange—it’s odd that it reappears after a few days.
As you also mentioned, I believe the bug related to the cloudflare-record-comment in the Cloudflare provider was already fixed in PR #5828

While it could be a new bug, is there a chance that the container image contents have changed?
I’m curious whether the SHA256 digest was the same between when it was working and when the issue reappeared.

[kinseii]

In your environment, when Istio and Ingress-nginx are running simultaneously, does External-DNS encounter this issue, or does having both running together actually prevent the problem?

@u-kai the cluster where the problems occurred had both Istio and Ingress nginx. I tested on another cluster that only had Ingress nginx and there were no problems. So I'm almost certain that this is a problem caused by Istio.

[ritik872000]

My cluster does not have istio, still the problem is there, can you share your config?

…

On Wed, 8 Oct, 2025, 12:59 pm kinseii, ***@***.***> wrote: *kinseii* left a comment (kubernetes-sigs/external-dns#5884) <#5884 (comment)> In your environment, when Istio and Ingress-nginx are running simultaneously, does External-DNS encounter this issue, or does having both running together actually prevent the problem? The cluster where the problems occurred had both Istio and Ingress nginx. I tested on another cluster that only had Ingress nginx and there were no problems. So I'm almost certain that this is a problem caused by Istio. — Reply to this email directly, view it on GitHub <#5884 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMUJ5HA3O36KPQFSTYCTXNL3WS4L7AVCNFSM6AAAAACIEPNT7KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOBQGEYTMNZUGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[u-kai]

@kinseii

Great, thanks for the details!

If possible, could you share your Istio configuration (the Helm values.yaml) for the cluster where the issue reproduces?

Also, could you double-check that cloudflare-record-comment isn’t set anywhere? External-DNS can pick this up either from its args or from resource annotations.

If you can confirm it’s absent from both the External-DNS deployment and all Ingress/Service resources in that cluster, that would help narrow this down.