DNS challenge requires several CNAME in some cases

[daschg]

Welcome

• Yes, I'm using a binary release within 2 latest releases.
• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've included all information below (version, config, etc).

What did you expect to see?

In my setup, the dns entries cannot be created via API. A second domain is served by cloudflare, where records can be created via API Calls.0
Acme supports cname alias to have a different zone serving TXT records for the challenge.
This is not working in lego with cloudflare when the dns zone for the challenge is different than the domain certs are requested.
Example:
Domain to request cert: test1.local.mydomain.de
Domain for challenge: challenge-domain.eu
Cname for _acme-challenge.local.mydomain.de is challenge-domain.eu

#dig +nocmd _acme-challenge.local.mydomain.de cname +noall +answer
_acme-challenge.local.mydomain.de. 150 IN CNAME  challenge-domain.eu.

challenge-domain.eu is the only zone setup up in cloudflare.

This setup works perfect with acme.sh because there is an option to pass the challenge domain

# ./acme.sh --log --issue -d test1.local.mydomain.de --challenge-alias challenge-domain.de --dns dns_cf
[Do 2. Jan 12:43:10 UTC 2025] Using CA: https://acme.zerossl.com/v2/DV90
[Do 2. Jan 12:43:10 UTC 2025] Single domain='test1.local.mydomain.de'
[Do 2. Jan 12:43:14 UTC 2025] Getting webroot for domain='test1.local.mydomain.de'
[Do 2. Jan 12:43:14 UTC 2025] Adding TXT value: DTXXXXXW0ZCkU3PakmP3Ub5WV-D46_rI9Ux7JAz9qZM for domain: _acme-challenge.challenge-domain.de
[Do 2. Jan 12:43:17 UTC 2025] Adding record
[Do 2. Jan 12:43:18 UTC 2025] Added, OK
[Do 2. Jan 12:43:18 UTC 2025] The TXT record has been successfully added.
[Do 2. Jan 12:43:18 UTC 2025] Let's check each DNS record now. Sleeping for 20 seconds first.
[Do 2. Jan 12:43:39 UTC 2025] You can use '--dnssleep' to disable public dns checks.
[Do 2. Jan 12:43:39 UTC 2025] See: https://github.com/acmesh-official/acme.sh/wiki/dnscheck
[Do 2. Jan 12:43:39 UTC 2025] Checking test1.local.mydomain.de for _acme-challenge.challenge-domain.de
[Do 2. Jan 12:43:39 UTC 2025] Success for domain test1.local.mydomain.de '_acme-challenge.challenge-domain.de'.
[Do 2. Jan 12:43:39 UTC 2025] All checks succeeded
[Do 2. Jan 12:43:39 UTC 2025] Verifying: test1.local.mydomain.de
[Do 2. Jan 12:43:40 UTC 2025] Processing. The CA is processing your order, please wait. (1/30)
[Do 2. Jan 12:43:43 UTC 2025] The retryafter=86400 value is too large (> 600), will not retry anymore.
[Do 2. Jan 12:43:43 UTC 2025] Removing DNS records.
[Do 2. Jan 12:43:43 UTC 2025] Removing txt: DTXXXXXW0ZCkU3PakmP3Ub5WV-D46_rI9Ux7JAz9qZM for domain: _acme-challenge.challenge-domain.de
[Do 2. Jan 12:43:47 UTC 2025] Successfully removed

Lego seems to use the top-level domain when looking up the zone in cloudflare leading to an error

# docker run --rm -e CLOUDFLARE_DNS_API_TOKEN=<redacted key> goacme/lego -a --email "<redacted email>" --domains "test.auth.mydomain.de" --dns cloudflare run
2025/01/02 11:22:20 No key found for account <redacted email>. Generating a P256 key.
2025/01/02 11:22:20 Saved key to /.lego/accounts/acme-v02.api.letsencrypt.org/<redacted email>/keys/<redacted email>.key
2025/01/02 11:22:20 [DEBUG] GET https://acme-v02.api.letsencrypt.org/directory
2025/01/02 11:22:20 [INFO] acme: Registering account for <redacted email>
2025/01/02 11:22:20 [DEBUG] HEAD https://acme-v02.api.letsencrypt.org/acme/new-nonce
2025/01/02 11:22:20 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/new-acct
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2025/01/02 11:22:20 [INFO] [test.auth.mydomain.de] acme: Obtaining bundled SAN certificate
2025/01/02 11:22:20 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/new-order
2025/01/02 11:22:21 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] acme: Could not find solver for: tls-alpn-01
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] acme: Could not find solver for: http-01
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] acme: use dns-01 solver
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] acme: Preparing to solve DNS-01
2025/01/02 11:22:22 [INFO] [test.auth.mydomain.de] acme: Cleaning DNS-01 challenge
2025/01/02 11:22:23 [WARN] [test.auth.mydomain.de] acme: cleaning up failed: cloudflare: failed to find zone mydomain.de.: zone could not be found
2025/01/02 11:22:23 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:23 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:23 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:23 Could not obtain certificates:
        error: one or more domains had a problem:
[test.auth.mydomain.de] [test.auth.mydomain.de] acme: error presenting token: cloudflare: failed to find zone mydomain.de.: zone could not be found

What did you see instead?

expectation is that the TXT record is created for _acme-challenge.challenge-domain.de in cloudflare and not for mydomain.de

How do you use lego?

Docker image

Reproduction steps

1. setup a cname for _acme-challenge.local.mydomain.de. with value challenge-domain.eu.
2. setup a second domain in cloudflare
3. create api key with read and edit permissions to the zone
4. run lego with cloudflare dns provider

Version of lego

lego version 4.21.0 linux/amd64

Logs

# paste output here
# docker run --rm -e CLOUDFLARE_DNS_API_TOKEN=<redacted key> goacme/lego -a --email "<redacted email>" --domains "test.auth.mydomain.de" --dns cloudflare run
2025/01/02 11:22:20 No key found for account <redacted email>. Generating a P256 key.
2025/01/02 11:22:20 Saved key to /.lego/accounts/acme-v02.api.letsencrypt.org/<redacted email>/keys/<redacted email>.key
2025/01/02 11:22:20 [DEBUG] GET https://acme-v02.api.letsencrypt.org/directory
2025/01/02 11:22:20 [INFO] acme: Registering account for <redacted email>
2025/01/02 11:22:20 [DEBUG] HEAD https://acme-v02.api.letsencrypt.org/acme/new-nonce
2025/01/02 11:22:20 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/new-acct
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2025/01/02 11:22:20 [INFO] [test.auth.mydomain.de] acme: Obtaining bundled SAN certificate
2025/01/02 11:22:20 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/new-order
2025/01/02 11:22:21 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] acme: Could not find solver for: tls-alpn-01
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] acme: Could not find solver for: http-01
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] acme: use dns-01 solver
2025/01/02 11:22:21 [INFO] [test.auth.mydomain.de] acme: Preparing to solve DNS-01
2025/01/02 11:22:22 [INFO] [test.auth.mydomain.de] acme: Cleaning DNS-01 challenge
2025/01/02 11:22:23 [WARN] [test.auth.mydomain.de] acme: cleaning up failed: cloudflare: failed to find zone mydomain.de.: zone could not be found
2025/01/02 11:22:23 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:23 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:23 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/authz/2146941123/123472384365
2025/01/02 11:22:23 Could not obtain certificates:
        error: one or more domains had a problem:
[test.auth.mydomain.de] [test.auth.mydomain.de] acme: error presenting token: cloudflare: failed to find zone mydomain.de.: zone could not be found

Go environment (if applicable)

$ go version && go env
# paste output here

[ldez]

Hello,

lego follows the CNAME by default (there is no specificity for Cloudflare).

The problem is related either to your DNS records (multiple CNAME, CNAME wildcard, invalid A record, etc.) or to your local network (firewall, etc.).

So, check your DNS records, and try to use --dns.resolvers

https://go-acme.github.io/lego/usage/cli/options/index.html

[daschg]

thanks for your reply.
Is my cname record as expected by lego ?
_acme-challenge.local.mydomain.de. 150 IN CNAME challenge-domain.eu.
Or is lego lexpecting it at the top level _acme-challenge.mydomain.de. ?
I just want to have the sub-domain served by the second domain

Since the scenario is working with acme.sh there must be a difference in the behavior of the two tools

[ldez]

lego expects a classic CNAME, nothing special, no apex (top-level) requirement.

But challenge-domain.eu. must be a real zone (i.e. it should have valid answer to SOA call)

Have you tried --dns.resolvers?

[daschg]

yes i tried with --dns.resolvers.

I did a further test and request a cert only for local.mydomain.de -> this works !

# docker run --rm -e CLOUDFLARE_DNS_API_TOKEN=<redacted key> goacme/lego -a --email "<redacted email>" --domains "local.mydomain.de" --dns cloudflare --dns.resolvers 1.1.1.1 run
2025/01/02 14:40:19 No key found for account <redacted email>. Generating a P256 key.
2025/01/02 14:40:19 Saved key to /.lego/accounts/acme-v02.api.letsencrypt.org/<redacted email>/keys/<redacted email>.key
2025/01/02 14:40:19 [DEBUG] GET https://acme-v02.api.letsencrypt.org/directory
2025/01/02 14:40:20 [INFO] acme: Registering account for <redacted email>
2025/01/02 14:40:20 [DEBUG] HEAD https://acme-v02.api.letsencrypt.org/acme/new-nonce
2025/01/02 14:40:20 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/new-acct
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2025/01/02 14:40:20 [INFO] [local.mydomain.de] acme: Obtaining bundled SAN certificate
2025/01/02 14:40:20 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/new-order
2025/01/02 14:40:20 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/authz/211234965/4544123485
2025/01/02 14:40:20 [INFO] [local.mydomain.de] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz/211234965/4544123485
2025/01/02 14:40:20 [INFO] [local.mydomain.de] acme: Could not find solver for: tls-alpn-01
2025/01/02 14:40:20 [INFO] [local.mydomain.de] acme: Could not find solver for: http-01
2025/01/02 14:40:20 [INFO] [local.mydomain.de] acme: use dns-01 solver
2025/01/02 14:40:20 [INFO] [local.mydomain.de] acme: Preparing to solve DNS-01
2025/01/02 14:40:20 [INFO] Found CNAME entry for "_acme-challenge.local.mydomain.de.": "challenge-domain.eu."
2025/01/02 14:40:22 [INFO] cloudflare: new record for local.mydomain.de, ID 18b123456b352e8c44f871bfba2c8d51
2025/01/02 14:40:22 [INFO] [local.mydomain.de] acme: Trying to solve DNS-01
2025/01/02 14:40:22 [INFO] Found CNAME entry for "_acme-challenge.local.mydomain.de.": "challenge-domain.eu."
2025/01/02 14:40:22 [INFO] [local.mydomain.de] acme: Checking DNS record propagation. [nameservers=1.1.1.1:53]
2025/01/02 14:40:24 [INFO] Wait for propagation [timeout: 2m0s, interval: 2s]
2025/01/02 14:40:24 [INFO] [local.mydomain.de] acme: Waiting for DNS record propagation.
2025/01/02 14:40:26 [INFO] [local.mydomain.de] acme: Waiting for DNS record propagation.
2025/01/02 14:40:28 [INFO] [local.mydomain.de] acme: Waiting for DNS record propagation.
2025/01/02 14:40:30 [INFO] [local.mydomain.de] acme: Waiting for DNS record propagation.
2025/01/02 14:40:32 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/chall/211234965/4544123485/R58NMw
2025/01/02 14:40:32 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/authz/211234965/4544123485
2025/01/02 14:40:40 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/authz/211234965/4544123485
2025/01/02 14:40:40 [INFO] [local.mydomain.de] The server validated our request
2025/01/02 14:40:40 [INFO] [local.mydomain.de] acme: Cleaning DNS-01 challenge
2025/01/02 14:40:40 [INFO] Found CNAME entry for "_acme-challenge.local.mydomain.de.": "challenge-domain.eu."
2025/01/02 14:40:41 [INFO] [local.mydomain.de] acme: Validations succeeded; requesting certificates
2025/01/02 14:40:41 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/finalize/2147243965/339985454935
2025/01/02 14:40:42 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/cert/034f4f812d47a47b76e4a91293dd26f1c8e6
2025/01/02 14:40:42 [DEBUG] POST https://acme-v02.api.letsencrypt.org/acme/cert/034f4f812d47a47b76e4a91293dd26f1c8e6/1
2025/01/02 14:40:43 [INFO] [local.mydomain.de] Server responded with a certificate.

However, i cannot get certs for subdomains e.g. test.local.mydomain.de

# docker run --rm -e CLOUDFLARE_DNS_API_TOKEN=<redacted key> goacme/lego -a --email "<redacted email>" --domains "test.local.mydomain.de" --dns cloudflare --dns.re
solvers 1.1.1.1  --server=https://acme-staging-v02.api.letsencrypt.org/directory run
2025/01/02 14:53:43 No key found for account <redacted email>. Generating a P256 key.
2025/01/02 14:53:43 Saved key to /.lego/accounts/acme-staging-v02.api.letsencrypt.org/<redacted email>/keys/<redacted email>.key
2025/01/02 14:53:43 [DEBUG] GET https://acme-staging-v02.api.letsencrypt.org/directory
2025/01/02 14:53:43 [INFO] acme: Registering account for <redacted email>
2025/01/02 14:53:43 [DEBUG] HEAD https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce
2025/01/02 14:53:43 [DEBUG] POST https://acme-staging-v02.api.letsencrypt.org/acme/new-acct
!!!! HEADS UP !!!!

Your account credentials have been saved in your Let's Encrypt
configuration directory at "/.lego/accounts".

You should make a secure backup of this folder now. This
configuration directory will also contain certificates and
private keys obtained from Let's Encrypt so making regular
backups of this folder is ideal.
2025/01/02 14:53:44 [INFO] [test.local.mydomain.de] acme: Obtaining bundled SAN certificate
2025/01/02 14:53:44 [DEBUG] POST https://acme-staging-v02.api.letsencrypt.org/acme/new-order
2025/01/02 14:53:44 [DEBUG] POST https://acme-staging-v02.api.letsencrypt.org/acme/authz/123453324/1234534634
2025/01/02 14:53:44 [INFO] [test.local.mydomain.de] AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz/123453324/1234534634
2025/01/02 14:53:44 [INFO] [test.local.mydomain.de] acme: Could not find solver for: tls-alpn-01
2025/01/02 14:53:44 [INFO] [test.local.mydomain.de] acme: Could not find solver for: http-01
2025/01/02 14:53:44 [INFO] [test.local.mydomain.de] acme: use dns-01 solver
2025/01/02 14:53:44 [INFO] [test.local.mydomain.de] acme: Preparing to solve DNS-01
2025/01/02 14:53:45 [INFO] [test.local.mydomain.de] acme: Cleaning DNS-01 challenge
2025/01/02 14:53:46 [WARN] [test.local.mydomain.de] acme: cleaning up failed: cloudflare: failed to find zone mydomain.de.: zone could not be found
2025/01/02 14:53:46 [DEBUG] POST https://acme-staging-v02.api.letsencrypt.org/acme/authz/123453324/1234534634
2025/01/02 14:53:46 [INFO] Deactivating auth: https://acme-staging-v02.api.letsencrypt.org/acme/authz/123453324/1234534634
2025/01/02 14:53:46 [DEBUG] POST https://acme-staging-v02.api.letsencrypt.org/acme/authz/123453324/1234534634
2025/01/02 14:53:46 Could not obtain certificates:
        error: one or more domains had a problem:
[test.local.mydomain.de] [test.local.mydomain.de] acme: error presenting token: cloudflare: failed to find zone mydomain.de.: zone could not be found

There the line 'Found CNAME entry for ..' is missing and it errors out with failed to find zone mydomain.de.

[ldez]

However, i cannot get certs for subdomains e.g. test.local.mydomain.de

Do you have a CNAME for _acme-challenge.test.local.mydomain.de?

[daschg]

No.
The cname is created manually and the reason to use two domains is the ability to automate entries in the second domain.

[ldez]

To handle the domain test.local.mydomain.de you need a CNAME on _acme-challenge.test.local.mydomain.de.

_acme-challenge.local.mydomain.de. 150 IN CNAME  challenge-domain.eu.
_acme-challenge.test.local.mydomain.de. 150 IN CNAME  challenge-domain.eu.

[daschg]

letsencrypt seems to travel up and and looks for a cname at local.mydomain.de
At least it is possible to get test.local.mydomain.de certificate with acme.sh and _acme-challenge.local.mydomain.de cname as one can see from my initial listing

Do you think it is possible to implement a similar behavior in lego ?
Otherwise the whole use-case of delegating the challenge to another domain makes no sense when a cname for each sub-domain is required

[ldez]

Otherwise the whole use-case of delegating the challenge to another domain makes no sense when a cname for each sub-domain is required

This is how it works: https://letsencrypt.org/2019/10/09/onboarding-your-customers-with-lets-encrypt-and-acme/#the-advantages-of-a-cname

You are trying to handle multiple subdomains: remember that *.*.example.com (wildcard of wildcard) is not something supported by the current ACME challenges.

Do you think it is possible to implement a similar behavior in lego ?

You are the first person who wants something like that, before adding an option (if it's possible) and handling the maintenance related to that, we need to have more than one user.
This is a ratio between usage and maintenance.

Also, lego is used as a library and should handle thousands of domains, acme.sh doesn't have this constraint as this is just a CLI.

[ldez]

🤔

I think you are misusing the DNS challenge: the main goal of a DNS challenge is to have a wildcard certificate.

You should use the following arguments to have a wildcard certificate:

--domains "*.local.example.com" --domains "local.example.com"

The certificate will handle:

• local.example.com
• test.local.example.com
• foo.local.example.com
• etc.

This works with only one CNAME: _acme-challenge.local.example.com.

[daschg]

Thanks for the input.
My understanding was that the dns challenge should be used for machines not exposed to the internet.
My goal is to create valid certificates for services in my local network. Distribute a wildcard certificate to all machines and services seems not the best approach.

You are the first person who wants something like that, before adding an option (if it's possible) and handling the maintenance related to that, we need to have more than one user. This is a ratio between usage and maintenance.

understood - maybe others will stumble over the same issue

Also, lego is used as a library and should handle thousands of domains, acme.sh doesn't have this constraint as this is just a CLI.

Yes - I would love to use lego in combination with traefik for my use cases.

Other options would be to transfer the whole domain mydomain.de to cloudflare, create a zone local.mydomain.de and an api token with just permissions for that zone.

[ldez]

My understanding was that the dns challenge should be used for machines not exposed to the internet.

Yes and no: if you are not exposing your domains you can use a local ACME server like smallstep

Currently, the DNS challenge is the only challenge that allows wildcard certificates.