Increased build time with `autosummary` templates

[tupui]

In some cases, using autosummary templates causes a significant increase in build time. This is particularly true for the scipy and nump docs (link to autosummary templates scipy uses).

I have verified this using this version of SciPy scipy/scipy#16660 and built the documentation on main and also with changes from #878. In both cases I run with and without templates.

originally discussed in #878,

[choldgraf]

Thanks for opening this! Can you try one other thing:

• Build with the autosummary template
• Count how many extra pages are created as part of this in the sidebar
• Remove the autosummary template
• Manually add that number of pages to the docs
• Re-build and see if it takes the same length

The goal here would be to test whether this is related to autosummary specifically, or just because of the large number of "stub files" that autosummary generates.

• Cross-ref for the original issue where this was reported: Performance regression with v0.10rc2 sidebar links for scipy and arrow docs #855

[tupui]

@QuLogic @timhoffm I see that for Matplotlib you have some templates and are using latest. Did you notice any build time increase?

[timhoffm]

I cannot really say. Matplotlib doc build times are annoyingly slow (up to 12min). They vary quite strongly for local builds depending on clean/non-clean builds, the environment I use (likely versions of sphinx, sphinx-gallery, pydata-sphinx-theme, python version, ...), other unknown factors. There's too much noise in the data, so that I cannot say off the top of my head whether latest pydata-sphinx-theme makes a difference.

Maybe @QuLogic has some statistics from CI runs?

[tupui]

Thanks for the feedback. Interesting because for SciPy starting from 0.10 it's doubling our build time.

[choldgraf]

Hey all - the latest state of getting around this is documented at this docs link:

• https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/performance.html#build-performance-and-size

I don't think it's on the roadmap of this project to make any more technical changes to address this, unless the suggestions above really don't work for many people.

[tupui]

I am sorry but I am a bit confused here as the issue is not linked to the TOC.

Both NumPy and SciPy have seen their build time literally double making the CI go above the hour. Until this is resolve we are stuck with 0.9 and cannot update.

If now you are saying this project won't do anything about the build issues with templates, then this should be documented and on our side we need to not use templates anymore so we can update the theme.

[choldgraf]

I think (?) It is related to the sidebar TOC. The problem is that this theme nests all of it's sub-pages in the sidebar dropdowns. When there is an extension like autodoc that generates thousands of extra pages then it means that we now have n_pages + thousands_of_autodoc_pages * n_pages + thousands_of_autodoc_pages references to resolve and this is what really slows down the builds. Am I not understanding this issue properly?

[tupui]

No we limit to 2 levels on the ref pages.

I will try again tomorrow to build with our newest version as the toc is even empty now.

[lwasser]

commenting here to be able to watch this thread. i'm going to test out templates as well and will see what happens to our build (our docs are tiny however compared to what @tupui is dealing with).

[choldgraf]

Just a thought @tupui it would be helpful if you provided more context and a minimum reproducible example in the top comment. I know you have cross linked to some discussion in the PR, but it makes it easier to read and understand if it's all contained within the relevant issue. For example, what are autosummary templates? How can someone add them themselves to test it out? What's the simplest template that still causes a slowdown?

One other thing that might be helpful. Check out this profile job in our noxfile:

https://github.com/pydata/pydata-sphinx-theme/blob/main/noxfile.py#L122

You might be able to repurpose that for the scipy docs to see if we can pinpoint the function that is taking up all the extra time.

[tupui]

I updated the linked PR to build with the latest version of the theme. It's still timing out and the build time goes above 40 mins.

Sorry I won't spend more time repeating what I said elsewhere and on discord. I already gave a reproducer. Yes it's big as it's SciPy's doc (still, @drammock was able to reproduce), also, as I said in the linked issue, I could not reproduce with a smaller chunk of SciPy's doc.

And as I said multiple times, I am not at all a web dev nor Sphinx expert. I just know my way around to make things work or adjust a bit according to my needs. I already spent a considerable amount of time on this issue and from all my report I think I showed enough effort...

One other thing that might be helpful. Check out this profile job in our noxfile:

You might be able to repurpose that for the scipy docs to see if we can pinpoint the function that is taking up all the extra time.

I don't get what I would be supposed to do here.

[choldgraf]

Sorry I won't spend more time repeating what I said elsewhere and on discord.

Sorry, I am just providing suggestions to make this issue easier for others to understand and take action on, in the hopes that it more quickly resolves your problem. I appreciate the scipy docs have this problem but I think a minimal reproducible example would be much more realistic for somebody to act on. I understand if you don't have time, maybe somebody else can figure out the minimal sphinx setup that reproduces this.

We are all volunteers here, working on nights and weekends to maintain this theme, so the more complex something is, the lower the chance that people have time to get to it.

I don't get what I would be supposed to do here.

I think the relevant line is here:

pydata-sphinx-theme/noxfile.py

Lines 167 to 169 in 2d245c9

	session.run(
	*f"py-spy record -o {output} -- sphinx-build {path_tmp} {path_tmp_out}".split() # noqa
	)

This uses py-spy to profile the build process of sphinx. It outputs an SVG that shows what things took the most time. You or somebody else could run this once with an old version of the theme, and once with a new version of the theme, and compare the two.

[tupui]

We are all volunteers here, working on nights and weekends to maintain this theme, so the more complex something is, the lower the chance that people have time to get to it.

We are all on the same wagon 😉 I don't think you need to use this argument with me... Hopefully the new CZI grant will help to mitigate that.

I think I did my fair share by advocating for using this theme in all places and helped raised a few things and made PRs when I could and I did my best so far with this specific issue. But it's past 4 versions now that both SciPy and NumPy cannot use. Hence, I am not sure why you are not considering this to have a higher priority on the list.