Get ReadTheDocs build to work without bursting memory limit when using sphinx autodoc #167
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add raven to sys.path to avoid having to full install raven for autodoc. Full install of raven will burst memory limit on ReadTheDocs.
Only necessary subset to build docs, to avoid bursting memory limit on ReadTheDocs.
Minimun dependencies are from Conda env. The rest of the extra dependencies (ex: numpy, xarray) are mocked.
…conv sphinx-autoapi was somehow conflicting with sphinx.
WARNING: autodoc: failed to import process 'processes.ObjectiveFunctionProcess' from module 'raven'; the following exception was raised:
Traceback (most recent call last):
File "/home/lvu/.conda/envs/raven/lib/python3.6/site-packages/sphinx/ext/autodoc/importer.py", line 36, in import_module
__import__(modname)
File "/zstore/repos/raven/raven/__init__.py", line 5, in <module>
from .wsgi import application
File "/zstore/repos/raven/raven/wsgi.py", line 4, in <module>
from .processes import processes
File "/zstore/repos/raven/raven/processes/__init__.py", line 48, in <module>
TSStatsProcess(),
File "/zstore/repos/raven/raven/processes/base_xclim.py", line 63, in __init__
inputs=self.load_inputs(eval(attrs['parameters'])),
TypeError: eval() arg 1 must be a string, bytes or code object
…install
WARNING: autodoc: failed to import process 'processes.RavenProcess' from module 'raven'; the following exception was raised:
Traceback (most recent call last):
File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/conda/fix-rtd-build-with-smaller-env-yml/lib/python3.6/site-packages/sphinx/ext/autodoc/importer.py", line 36, in import_module
__import__(modname)
File "/home/docs/checkouts/readthedocs.org/user_builds/pavics-raven/checkouts/fix-rtd-build-with-smaller-env-yml/raven/__init__.py", line 15, in <module>
raise IOError("The raven executable is not installed.")
OSError: The raven executable is not installed.
tlvu
added a commit
to bird-house/cookiecutter-birdhouse
that referenced
this pull request
May 26, 2020
This was needed with the minimal environment-docs.yml file. Should have been part of the PR that added the environment-docs.yml file. See PR Ouranosinc/raven#167 (commit Ouranosinc/raven@949ff60) for a real life usage of mock import.
Zeitsperre
pushed a commit
to bird-house/cookiecutter-birdhouse
that referenced
this pull request
Jan 5, 2021
This was needed with the minimal environment-docs.yml file. Should have been part of the PR that added the environment-docs.yml file. See PR Ouranosinc/raven#167 (commit Ouranosinc/raven@949ff60) for a real life usage of mock import.
Zeitsperre
pushed a commit
that referenced
this pull request
Aug 17, 2023
…v-yml Get ReadTheDocs build to work without bursting memory limit when using sphinx autodoc. * New `environment-docs.yml` with minimal package install required to be used when building docs on ReadTheDocs so we do not burst the memory limit. * Mock import all the 3rd party dependency packages (xarray, numpy, ...) so these dependencies are not needed to be installed to not burst the memory limit. * Raven source is added to sys.path so it can be "crawled" by autodoc without being installed. * Migrated .readthedocs.yml config file to v2 format, not sure if it helped with the memory limitation but v2 is currently the latest version and we seem to be using v1. * New `DO_NOT_CHECK_EXECUTABLE_EXISTENCE` env var supported by raven to not break doc build. Should technically not be used in other context than doc build. * Upgraded xclim version used by production raven build, not just doc build. * Fix TravisCI doc build by adding pandoc to environment.yml.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This PR fixes #148. No need to use Github pages anymore.
Working doc build with autodoc: https://pavics-raven.readthedocs.io/en/fix-rtd-build-with-smaller-env-yml/processes.html
Changes:
New
environment-docs.ymlwith minimal package install required to be used when building docs on ReadTheDocs so we do not burst the memory limit.Mock import all the 3rd party dependency packages (xarray, numpy, ...) so these dependencies are not needed to be installed to not burst the memory limit.
Raven source is added to sys.path so it can be "crawled" by autodoc without being installed.
Migrated .readthedocs.yml config file to v2 format, not sure if it helped with the memory limitation but v2 is currently the latest version and we seem to be using v1.
New
DO_NOT_CHECK_EXECUTABLE_EXISTENCEenv var supported by raven to not break doc build. Should technically not be used in other context than doc build.Upgraded xclim version used by production raven build, not just doc build.
Fix TravisCI doc build by adding pandoc to environment.yml.