Skip to content

Conversation

@sharkdp
Copy link
Contributor

@sharkdp sharkdp commented Sep 10, 2025

Summary

We use classes like _typeshed._type_checker_internals.NamedTupleFallback to tack on additional attributes/methods to instances of user-defined NamedTuples (or TypedDicts), even though these classes are not present in the MRO of those types.

The problem is that those classes use implicit and explicit Self annotations which refer to NamedTupleFallback itself, instead of to the actual type that we're adding those methods to:

class NamedTupleFallback(tuple[Any, ...]):
    # […]
    def _replace(self, **kwargs: Any) -> typing_extensions.Self: ...

In effect, when we access _replace on an instance of a custom NamedTuple instance, its self parameter and return type refer to the wrong Self. This leads to incorrect "Argument to bound method _replace is incorrect: Argument type Person does not satisfy upper bound NamedTupleFallback of type variable Self" errors on #18007. It would also lead to similar errors on TypedDicts, if they would already implement assignability properly.

Test Plan

I applied the following patch to typeshed and verified that no errors appear anymore.

diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi
index feb22aae00..8e41034f19 100644
--- a/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi
+++ b/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/_type_checker_internals.pyi
@@ -29,27 +29,27 @@ class TypedDictFallback(Mapping[str, object], metaclass=ABCMeta):
         __readonly_keys__: ClassVar[frozenset[str]]
         __mutable_keys__: ClassVar[frozenset[str]]
 
-    def copy(self) -> typing_extensions.Self: ...
+    def copy(self: typing_extensions.Self) -> typing_extensions.Self: ...
     # Using Never so that only calls using mypy plugin hook that specialize the signature
     # can go through.
-    def setdefault(self, k: Never, default: object) -> object: ...
+    def setdefault(self: typing_extensions.Self, k: Never, default: object) -> object: ...
     # Mypy plugin hook for 'pop' expects that 'default' has a type variable type.
-    def pop(self, k: Never, default: _T = ...) -> object: ...  # pyright: ignore[reportInvalidTypeVarUse]
-    def update(self, m: typing_extensions.Self, /) -> None: ...
-    def __delitem__(self, k: Never) -> None: ...
-    def items(self) -> dict_items[str, object]: ...
-    def keys(self) -> dict_keys[str, object]: ...
-    def values(self) -> dict_values[str, object]: ...
+    def pop(self: typing_extensions.Self, k: Never, default: _T = ...) -> object: ...  # pyright: ignore[reportInvalidTypeVarUse]
+    def update(self: typing_extensions.Self, m: typing_extensions.Self, /) -> None: ...
+    def __delitem__(self: typing_extensions.Self, k: Never) -> None: ...
+    def items(self: typing_extensions.Self) -> dict_items[str, object]: ...
+    def keys(self: typing_extensions.Self) -> dict_keys[str, object]: ...
+    def values(self: typing_extensions.Self) -> dict_values[str, object]: ...
     @overload
-    def __or__(self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
+    def __or__(self: typing_extensions.Self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
     @overload
-    def __or__(self, value: dict[str, Any], /) -> dict[str, object]: ...
+    def __or__(self: typing_extensions.Self, value: dict[str, Any], /) -> dict[str, object]: ...
     @overload
-    def __ror__(self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
+    def __ror__(self: typing_extensions.Self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...
     @overload
-    def __ror__(self, value: dict[str, Any], /) -> dict[str, object]: ...
+    def __ror__(self: typing_extensions.Self, value: dict[str, Any], /) -> dict[str, object]: ...
     # supposedly incompatible definitions of __or__ and __ior__
-    def __ior__(self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...  # type: ignore[misc]
+    def __ior__(self: typing_extensions.Self, value: typing_extensions.Self, /) -> typing_extensions.Self: ...  # type: ignore[misc]
 
 # Fallback type providing methods and attributes that appear on all `NamedTuple` types.
 class NamedTupleFallback(tuple[Any, ...]):
@@ -61,18 +61,18 @@ class NamedTupleFallback(tuple[Any, ...]):
         __orig_bases__: ClassVar[tuple[Any, ...]]
 
     @overload
-    def __init__(self, typename: str, fields: Iterable[tuple[str, Any]], /) -> None: ...
+    def __init__(self: typing_extensions.Self, typename: str, fields: Iterable[tuple[str, Any]], /) -> None: ...
     @overload
     @typing_extensions.deprecated(
         "Creating a typing.NamedTuple using keyword arguments is deprecated and support will be removed in Python 3.15"
     )
-    def __init__(self, typename: str, fields: None = None, /, **kwargs: Any) -> None: ...
+    def __init__(self: typing_extensions.Self, typename: str, fields: None = None, /, **kwargs: Any) -> None: ...
     @classmethod
     def _make(cls, iterable: Iterable[Any]) -> typing_extensions.Self: ...
-    def _asdict(self) -> dict[str, Any]: ...
-    def _replace(self, **kwargs: Any) -> typing_extensions.Self: ...
+    def _asdict(self: typing_extensions.Self) -> dict[str, Any]: ...
+    def _replace(self: typing_extensions.Self, **kwargs: Any) -> typing_extensions.Self: ...
     if sys.version_info >= (3, 13):
-        def __replace__(self, **kwargs: Any) -> typing_extensions.Self: ...
+        def __replace__(self: typing_extensions.Self, **kwargs: Any) -> typing_extensions.Self: ...
 
 # Non-default variations to accommodate couroutines, and `AwaitableGenerator` having a 4th type parameter.
 _S = TypeVar("_S")

@sharkdp sharkdp added the ty Multi-file analysis & type inference label Sep 10, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 10, 2025

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

@github-actions
Copy link
Contributor

github-actions bot commented Sep 10, 2025

mypy_primer results

No ecosystem changes detected ✅
No memory usage changes detected ✅

@github-actions
Copy link
Contributor

github-actions bot commented Sep 10, 2025

ecosystem-analyzer results

No diagnostic changes detected ✅
Full report with detailed diff

@sharkdp sharkdp force-pushed the david/patch-self-on-fallback-types branch from fbfc61f to 093c576 Compare September 12, 2025 12:42
@sharkdp sharkdp changed the title [ty] Patch (implicit) Self on fallback types [ty] Patch Self on fallback types Sep 12, 2025
@sharkdp sharkdp force-pushed the david/patch-self-on-fallback-types branch from 093c576 to 92985c6 Compare September 12, 2025 13:26
@sharkdp sharkdp changed the title [ty] Patch Self on fallback types [ty] Patch Self for fallback-methods on NamedTuples and TypedDicts Sep 12, 2025
@sharkdp sharkdp marked this pull request as ready for review September 12, 2025 13:27
@carljm carljm removed their request for review September 12, 2025 21:41
@sharkdp sharkdp force-pushed the david/patch-self-on-fallback-types branch from e49a2f9 to 9649de7 Compare September 15, 2025 11:05
@sharkdp
Copy link
Contributor Author

sharkdp commented Sep 15, 2025

I will merge this without approval, because I want to have an easier way of rebasing #18007 on this. Happy to address any post-merge comments.

@sharkdp sharkdp merged commit 25cbf38 into main Sep 15, 2025
38 checks passed
@sharkdp sharkdp deleted the david/patch-self-on-fallback-types branch September 15, 2025 14:21
@dcreager
Copy link
Member

I will merge this without approval

👍 Looks good!

sharkdp added a commit that referenced this pull request Sep 15, 2025
## Summary

This mainly removes an internal inconsistency, where we didn't remove
the `Self` type variable when eagerly binding `Self` to an instance
type. It has no observable effect, apparently.

builds on top of #20328

## Test Plan

None
sharkdp added a commit that referenced this pull request Sep 22, 2025
## Summary

Part of astral-sh/ty#159

This PR only adjusts the signature of a method so if it has a `self`
argument then that argument will have type of `Typing.Self` even if it's
not specified. If user provides an explicit annotation then Ty will not
override that annotation.

## Follow Ups

- astral-sh/ty#1131
- astral-sh/ty#1157
- astral-sh/ty#1156
- astral-sh/ty#1173
- #20328
- astral-sh/ty#1163
- astral-sh/ty#1196

## Test Plan

Added mdtests.
Also some tests need #18473 to
work completely. So I added a todo for those new cases that I added.

---------

Co-authored-by: David Peter <[email protected]>
sharkdp added a commit that referenced this pull request Sep 23, 2025
Part of astral-sh/ty#159

This PR only adjusts the signature of a method so if it has a `self`
argument then that argument will have type of `Typing.Self` even if it's
not specified. If user provides an explicit annotation then Ty will not
override that annotation.

- astral-sh/ty#1131
- astral-sh/ty#1157
- astral-sh/ty#1156
- astral-sh/ty#1173
- #20328
- astral-sh/ty#1163
- astral-sh/ty#1196

Added mdtests.
Also some tests need #18473 to
work completely. So I added a todo for those new cases that I added.

---------

Co-authored-by: David Peter <[email protected]>
sharkdp added a commit that referenced this pull request Sep 24, 2025
Part of astral-sh/ty#159

This PR only adjusts the signature of a method so if it has a `self`
argument then that argument will have type of `Typing.Self` even if it's
not specified. If user provides an explicit annotation then Ty will not
override that annotation.

- astral-sh/ty#1131
- astral-sh/ty#1157
- astral-sh/ty#1156
- astral-sh/ty#1173
- #20328
- astral-sh/ty#1163
- astral-sh/ty#1196

Added mdtests.
Also some tests need #18473 to
work completely. So I added a todo for those new cases that I added.

---------

Co-authored-by: David Peter <[email protected]>
sharkdp added a commit that referenced this pull request Sep 25, 2025
Part of astral-sh/ty#159

This PR only adjusts the signature of a method so if it has a `self`
argument then that argument will have type of `Typing.Self` even if it's
not specified. If user provides an explicit annotation then Ty will not
override that annotation.

- astral-sh/ty#1131
- astral-sh/ty#1157
- astral-sh/ty#1156
- astral-sh/ty#1173
- #20328
- astral-sh/ty#1163
- astral-sh/ty#1196

Added mdtests.
Also some tests need #18473 to
work completely. So I added a todo for those new cases that I added.

---------

Co-authored-by: David Peter <[email protected]>
sharkdp added a commit that referenced this pull request Sep 25, 2025
Part of astral-sh/ty#159

This PR only adjusts the signature of a method so if it has a `self`
argument then that argument will have type of `Typing.Self` even if it's
not specified. If user provides an explicit annotation then Ty will not
override that annotation.

- astral-sh/ty#1131
- astral-sh/ty#1157
- astral-sh/ty#1156
- astral-sh/ty#1173
- #20328
- astral-sh/ty#1163
- astral-sh/ty#1196

Added mdtests.
Also some tests need #18473 to
work completely. So I added a todo for those new cases that I added.

---------

Co-authored-by: David Peter <[email protected]>
sharkdp added a commit that referenced this pull request Sep 29, 2025
Part of astral-sh/ty#159

This PR only adjusts the signature of a method so if it has a `self`
argument then that argument will have type of `Typing.Self` even if it's
not specified. If user provides an explicit annotation then Ty will not
override that annotation.

- astral-sh/ty#1131
- astral-sh/ty#1157
- astral-sh/ty#1156
- astral-sh/ty#1173
- #20328
- astral-sh/ty#1163
- astral-sh/ty#1196

Added mdtests.
Also some tests need #18473 to
work completely. So I added a todo for those new cases that I added.

---------

Co-authored-by: David Peter <[email protected]>
sharkdp added a commit that referenced this pull request Sep 29, 2025
Part of astral-sh/ty#159

This PR only adjusts the signature of a method so if it has a `self`
argument then that argument will have type of `Typing.Self` even if it's
not specified. If user provides an explicit annotation then Ty will not
override that annotation.

- astral-sh/ty#1131
- astral-sh/ty#1157
- astral-sh/ty#1156
- astral-sh/ty#1173
- #20328
- astral-sh/ty#1163
- astral-sh/ty#1196

Added mdtests.
Also some tests need #18473 to
work completely. So I added a todo for those new cases that I added.

---------

Co-authored-by: David Peter <[email protected]>
sharkdp added a commit that referenced this pull request Sep 29, 2025
Part of astral-sh/ty#159

This PR only adjusts the signature of a method so if it has a `self`
argument then that argument will have type of `Typing.Self` even if it's
not specified. If user provides an explicit annotation then Ty will not
override that annotation.

- astral-sh/ty#1131
- astral-sh/ty#1157
- astral-sh/ty#1156
- astral-sh/ty#1173
- #20328
- astral-sh/ty#1163
- astral-sh/ty#1196

Added mdtests.
Also some tests need #18473 to
work completely. So I added a todo for those new cases that I added.

---------

Co-authored-by: David Peter <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants