Autodoc typehints

[leezu]

Description

This adds sphinx autodoc based on Python 3 typehints.

Impact: Auto-generate (missing) type annotations in the documentation based on machine-readable typehints. The auto-generated type annotations look as follows: http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-830/4/api/modules/vocab.html#gluonnlp.Vocab

[mli]

Found link check problems in job PR-830/4:
(line 19) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 21) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 442) broken https://www.aclweb.org/anthology/P02-1040.pdf)[1 - 404 Client Error: Not Found for url: https://www.aclweb.org/anthology/P02-1040.pdf)%5B1
(line 92) broken https://nlp.stanford.edu/pubs/glove.pdf)[2 - 404 Client Error: NOT FOUND for url: https://nlp.stanford.edu/pubs/glove.pdf)%5B2
(line 208) broken https://www.bioinf.jku.at/publications/older/2604.pdf)[3 - 404 Client Error: Not Found for url: https://www.bioinf.jku.at/publications/older/2604.pdf)%5B3

[mli]

Job PR-830/4 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-830/4/index.html

[leezu]

@dmlc/gluon-nlp-team please take a look at above example and make your opinion known.

[JulianSlzr]

I think typing will add some automatic robustness and reduce unit test size. Some thoughts:

• Can we make from typing import * the convention and implicitly reserve its subclass names (List, Optional, etc.)? If that's too permissive we can do from typing import Optional, List, .... Both would be preferable to wordy type hints param: typing.Optional[typing.List[int, typing...]]
• How do we document tensor-shape constraints not in the type signature? We'd need a convention.
• Idea: Could typing / NDArray be extended to support tensor shape checking? e.g., def f(...) -> NDArray.dim(3)

[leezu]

Thanks @JulianSlzr for your thoughts.

• For the imports: from typing import * may be too permissive, but from typing import Optional, List, ... looks good to me.
• Shape convention: We can initially keep the current convention of documenting shapes in the free-text part.
• Shape typing: Let's address this in the MXNet. Here are some numpy related discussions https://github.com/numpy/numpy-stubs/issues/5 Type hinting / annotation (PEP 484) for ndarray, dtype, and ufunc numpy/numpy#7370 https://docs.google.com/document/d/1vpMse4c6DrWH5rq2tQSx3qwP_m_0lyn-Ij4WHqQqRHY/

[codecov]

Codecov Report

Merging #830 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #830      +/-   ##
==========================================
- Coverage   90.18%   90.17%   -0.01%     
==========================================
  Files          66       66              
  Lines        6345     6344       -1     
==========================================
- Hits         5722     5721       -1     
  Misses        623      623

Impacted Files	Coverage Δ
src/gluonnlp/vocab/vocab.py	97.28% <100%> (-0.02%)	⬇️

[mli]

Found link check problems in job PR-830/5:
(line 19) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 5) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 21) broken https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/ - 404 Client Error: Not Found for url: https://www.salesforce.com/products/einstein/ai-research/the-wikitext-dependency-language-modeling-dataset/
(line 442) broken https://www.aclweb.org/anthology/P02-1040.pdf)[1 - 404 Client Error: Not Found for url: https://www.aclweb.org/anthology/P02-1040.pdf)%5B1
(line 92) broken https://nlp.stanford.edu/pubs/glove.pdf)[2 - 404 Client Error: NOT FOUND for url: https://nlp.stanford.edu/pubs/glove.pdf)%5B2
(line 208) broken https://www.bioinf.jku.at/publications/older/2604.pdf)[3 - 404 Client Error: Not Found for url: https://www.bioinf.jku.at/publications/older/2604.pdf)%5B3

[mli]

Job PR-830/5 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-830/5/index.html

[eric-haibin-lin]

Nice! I guess the next step is to apply these changes to all other API docs?

[mli]

Job PR-830/6 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-830/6/index.html

[mli]

Job PR-830/7 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-830/7/index.html