Automatically check for bad links

[kylixafonso]

Since the ADA tools link on the developer portal showcase returns a 404, there's a good chance other links do. Would be good to detect this automatically and fix. There's plenty of plugins and tools to set up against CI to check for bad links.

[katomm]

Great suggestion, there is a crawler running regularly and there are regular pull requests to fix broken links, for example #1486. If there is interest we can also publish the output so everybody can fix links. This is an ongoing effort of course.

Last run was on June 18ths with these outputs:

`Job Report for https://developers.cardano.org/
Start Time: 2025-06-18 10:03:28
End Time: 2025-06-18 10:33:11
Duration: 0:29:43
Crawling Summary
• Internal

• HTML: 2809 Files: 0
  • External
• HTML: 833 Files: 10
  • Total URLs visited: 4441
  ⚠️ Client Errors

1. From: https://developers.cardano.org/docs/get-started/lucid-evolution/get-started
   • To: https://anastasia-labs.github.io/lucid-evolution/documentation/deep-dives/make-payments|https://anastasia-labs.github.io/lucid-evolution/documentation/deep-dives/make-payments
   • Status: 404 - Not Found
2. From: https://developers.cardano.org/docs/get-started/lucid-evolution/get-started#installation
   • To: https://anastasia-labs.github.io/lucid-evolution/install|https://anastasia-labs.github.io/lucid-evolution/install
   • Status: 404 - Not Found
3. From: https://developers.cardano.org/tools?tags=koios
   • To: https://anastasia-labs.github.io/lucid-evolution/|https://anastasia-labs.github.io/lucid-evolution/
   • Status: 404 - Not Found
4. From: https://developers.cardano.org/tools?tags=koios
   • To: https://anastasia-labs.github.io/lucid-evolution/documentation/core-concepts/instantiate-evolution|https://anastasia-labs.github.io/lucid-evolution/documentation/core-concepts/instantiate-evolution
   • Status: 404 - Not Found
5. From: http://developers.cardano.org/docs/operate-a-stake-pool/relay-node-configuration#configuration-files
   • To: http://developers.cardano.org/docs/operate-a-stake-pool/docs/get-started/cardano-node/p2p|http://developers.cardano.org/docs/operate-a-stake-pool/docs/get-started/cardano-node/p2p
   • Status: 404 - Not Found
6. From: http://developers.cardano.org/docs/native-tokens/token-registry/cardano-token-registry-server
   • To: https://tokens.cardano.org/apidocs|https://tokens.cardano.org/apidocs
   • Status: 406 - Not Acceptable
7. From: http://developers.cardano.org/docs/native-tokens/token-registry/cardano-token-registry-cip68#cons-of-cip-68
   • To: https://www.crypto-news-flash.com/cardano-advances-blockchain-innovation-with-cip25-metadata-validation-boosting-nfts-and-amms/|https://www.crypto-news-flash.com/cardano-advances-blockchain-innovation-with-cip25-metadata-validation-boosting-nfts-and-amms/
   • Status: 404 - Not Found
8. From: http://developers.cardano.org/blog/launch-of-the-developer-portal-2021
   • To: https://cardanofoundation.org/nfta/|https://cardanofoundation.org/nfta/
   • Status: 404 - Not Found
9. From: http://developers.cardano.org/blog/nft-minting-standard
   • To: https://cardanofoundation.org/nfta|https://cardanofoundation.org/nfta
   • Status: 404 - Not Found
10. From: http://developers.cardano.org/blog/2022-05-30-may#integrating-with-milkomeda
    • To: https://dcspark.gitbook.io/milkomeda-getting-started/for-developers|https://dcspark.gitbook.io/milkomeda-getting-started/for-developers
    • Status: 404 - Not Found
11. From: https://developers.cardano.org/docs/operate-a-stake-pool/relay-node-configuration#configuration-files
    • To: https://developers.cardano.org/docs/operate-a-stake-pool/docs/get-started/cardano-node/p2p|https://developers.cardano.org/docs/operate-a-stake-pool/docs/get-started/cardano-node/p2p
    • Status: 404 - Not Found
    ⚠️ Server Errors
12. From: http://developers.cardano.org/blog/2021-12-20-december
    • To: https://cardanoassets.com/|https://cardanoassets.com/
    • Status: 526 - Invalid SSL Certificate
    ⚠️ Connection Errors
13. From: https://developers.cardano.org/docs/get-started/air-gap#finish--reboot
    • To: https://help.ubuntu.com/community/Grub2|https://help.ubuntu.com/community/Grub2
    • Details: HTTPSConnectionPool(host='help.ubuntu.com', port=443): Max retries exceeded with url: /community/Grub2 (Caused by ReadTimeoutError("HTTPSConnectionPool(host='help.ubuntu.com', port=443): Read timed out. (read timeout=5)"))
14. From: http://developers.cardano.org/docs/governance/project-catalyst#faq
    • To: https://cardanocataly.st/en/faq/|https://cardanocataly.st/en/faq/
    • Details: HTTPSConnectionPool(host='cardanocataly.st', port=443): Max retries exceeded with url: /en/faq/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fb34880b370>: Failed to establish a new connection: [Errno -2] Name or service not known'))
15. From: http://developers.cardano.org/blog/tags/payments
    • To: https://mstreet.app|https://mstreet.app/
    • Details: HTTPSConnectionPool(host='mstreet.app', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7fb33de128b0>, 'Connection to mstreet.app timed out. (connect timeout=5)'))`

[rphair]

awesome 😎 ... Can the script be modified so the link destinations are clickable... i.e. removing the extra | + repeated link (or separating the 2 links, if they're calculated differently)? This would make it easier to verify that the links are actually bad (at least one of the link problems above was temporary).