typing.Generator producing inconsistent violations

[walrusVision]

Testing Version: pydoclint-0.6.10

I was testing out pydoclint on an existing code base and I came across some inconsistent behaviours which I'm not sure are bugs, or me not understanding the reason for the violations, but which seem to be related to the use of typing.Generator

I was hoping if you clarify what is happening here for me please?

My test example is

import typing

def test_yield_with_typing(gen: typing.Iterable[typing.Any]) -> typing.Generator[typing.Any]:
    """
    Testing the behaviour of pydoclint with a function which has specified the Generator via
    the typing module.

    Args:
        gen: An iterable argument

    Yields:
        Generator[typing.Any]: The iterable argument

    """
    yield gen

Command: flake8 --select DOC typing_generator_test.py --style google --arg-type-hints-in-signature True --arg-type-hints-in-docstring False
Expected: This will produce no violations
Result: Produced four violations

• DOC201 Function test_with_yield_with_typing does not have a return section in docstring
• DOC203 Function test_with_yield_with_typing return type(s) in docstring not consistent with the return annotation. Return annotation has 1 type(s); docstring return section has 0 type(s).
• DOC403 Function test_with_yield_with_typing has a "Yields" section in the docstring, but there are no "yield" statements, or the return annotation is not a Generator/Iterator/Iterable. (Or it could be because the function lacks a return annotation.)
• DOC404 Function test_with_yield_with_typing yield type(s) in docstring not consistent with the return annotation. Return annotation does not exist or is not Generator[...]/Iterator[...]/Iterable[...], but docstring "yields" section has 1 type(s).

However if this same example changes typing.Generator to just Generator then it works as expected

import typing

def test_yield_with_typing(gen: typing.Iterable[typing.Any]) -> Generator[typing.Any]:
    """
    Testing the behaviour of pydoclint with a function which has specified the Generator via
    the typing module.

    Args:
        gen: An iterable argument

    Yields:
        Generator[typing.Any]: The iterable argument

    """
    yield gen

Command: flake8 --select DOC typing_generator_test.py --style google --arg-type-hints-in-signature True --arg-type-hints-in-docstring False
Expected: This will produce no violations
Result: Produced no violations

But I can also produce an example of a function which uses typing.Generator and correctly returns no violations

def test_yield_with_typing_no_args() -> typing.Generator[typing.Any]:
    """
    Testing the behaviour of pydoclint with a function which has specified the Generator via
    the typing module and has no arguments

    Yields:
        Generator[typing.Any]: An iterable argument

    """
    yield [1]

Command: flake8 --select DOC typing_generator_test.py --style google --arg-type-hints-in-signature True --arg-type-hints-in-docstring False
Expected: This will produce no violations
Result: Produced no violations

[jsh9]

Hi @walrusVision , thanks for raising this issue!

Here's my investigation result

• Your 1st example is expected to fail, because pydoclint does dumb string matching (in order to be fast), so it doesn't know typing.Generator means Generator (see this doc for more explainations)
• Your 2nd example is expected to pass without issues
• Your 3rd example should have triggered violations, but it didn't. This is a bug, and I'll fix it

So my recommendation is: go with what you did in your 2nd example

[jsh9]

Additionally, only Generator[blah blah blah] in the return type annotation will inform pydoclint that this is a function that doesn't/shouldn't have a return section in the docstring. pydoclint doesn't recognize typing.Generator[blah blah blah].

Again, this is because pydoclint does string matching here for speed.

I know some people use typing.Generator, but there are also some people who'd do:

import typing as ty

...

def test_yield_with_typing_no_args() -> ty.Generator[typing.Any]:

And some would even do import typing as typ.

There are so many variations, but the majority of people seem to gravitate towards:

from typing import Generator

...

def test_yield_with_typing_no_args() -> Generator[typing.Any]:

So that's why pydoclint only recognizes Generator.

[jsh9]

The fix is published in 0.6.11

[walrusVision]

Thanks for the explanation and the link!