[ty] Use declared variable types as bidirectional type context #20796
+102
−63
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ibraheemdev
requested review from
AlexWaygood,
dcreager and
sharkdp
as code owners
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-10-16 19:20:45.139893750 +0000
+++ new-output.txt 2025-10-16 19:20:48.626930815 +0000
@@ -883,6 +883,13 @@
tuples_type_form.py:15:1: error[invalid-assignment] Object of type `tuple[Literal[1], Literal[""]]` is not assignable to `tuple[int, int]`
tuples_type_form.py:25:1: error[invalid-assignment] Object of type `tuple[Literal[1]]` is not assignable to `tuple[()]`
tuples_type_form.py:36:1: error[invalid-assignment] Object of type `tuple[Literal[1], Literal[2], Literal[3], Literal[""]]` is not assignable to `tuple[int, ...]`
+typeddicts_operations.py:22:17: error[invalid-assignment] Invalid assignment to key "name" with declared type `str` on TypedDict `Movie`: value of type `Literal[1982]`
+typeddicts_operations.py:23:17: error[invalid-assignment] Invalid assignment to key "year" with declared type `int` on TypedDict `Movie`: value of type `Literal[""]`
+typeddicts_operations.py:24:7: error[invalid-key] Invalid key access on TypedDict `Movie`: Unknown key "other"
+typeddicts_operations.py:26:13: error[invalid-key] Invalid key access on TypedDict `Movie`: Unknown key "other"
+typeddicts_operations.py:28:9: error[missing-typed-dict-key] Missing required key 'year' in TypedDict `Movie` constructor
+typeddicts_operations.py:29:42: error[invalid-argument-type] Invalid argument to key "year" with declared type `int` on TypedDict `Movie`: value of type `float`
+typeddicts_operations.py:32:36: error[invalid-key] Invalid key access on TypedDict `Movie`: Unknown key "other"
typeddicts_operations.py:37:20: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
typeddicts_operations.py:62:1: error[unresolved-attribute] Type `MovieOptional` has no attribute `clear`
typeddicts_readonly.py:24:4: error[invalid-assignment] Cannot assign to key "members" on TypedDict `Band`: key is marked read-only
@@ -892,6 +899,9 @@
typeddicts_readonly.py:61:4: error[invalid-assignment] Cannot assign to key "year" on TypedDict `Movie2`: key is marked read-only
typeddicts_readonly_inheritance.py:36:4: error[invalid-assignment] Cannot assign to key "name" on TypedDict `Album2`: key is marked read-only
typeddicts_readonly_inheritance.py:65:19: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `RequiredName` constructor
+typeddicts_readonly_inheritance.py:82:14: error[invalid-assignment] Invalid assignment to key "ident" with declared type `str` on TypedDict `User`: value of type `Literal[3]`
+typeddicts_readonly_inheritance.py:83:15: error[invalid-argument-type] Invalid argument to key "ident" with declared type `str` on TypedDict `User`: value of type `Literal[3]`
+typeddicts_readonly_inheritance.py:84:5: error[missing-typed-dict-key] Missing required key 'ident' in TypedDict `User` constructor
typeddicts_type_consistency.py:69:21: error[invalid-key] Invalid key access on TypedDict `A3`: Unknown key "y"
typeddicts_type_consistency.py:101:1: error[invalid-assignment] Object of type `Unknown | None` is not assignable to `str`
typeddicts_type_consistency.py:126:56: error[invalid-argument-type] Invalid argument to key "inner_key" with declared type `str` on TypedDict `Inner1`: value of type `Literal[1]`
@@ -900,5 +910,5 @@
typeddicts_usage.py:28:17: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
typeddicts_usage.py:28:18: error[invalid-key] Invalid key access on TypedDict `Movie`: Unknown key "title"
typeddicts_usage.py:40:24: error[invalid-type-form] The special form `typing.TypedDict` is not allowed in type expressions. Did you mean to use a concrete TypedDict or `collections.abc.Mapping[str, object]` instead?
-Found 902 diagnostics
+Found 912 diagnostics
WARN A fatal error occurred while checking some files. Not all project files were analyzed. See the diagnostics list above for details. |
|
ibraheemdev
force-pushed
the
ibraheem/declared-type-context
branch
from
7beb24b to
0fbaae1
Compare
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-argument-type |
9 | 1 | 12 |
invalid-assignment |
0 | 4 | 9 |
missing-typed-dict-key |
10 | 0 | 0 |
possibly-missing-attribute |
0 | 1 | 3 |
redundant-cast |
2 | 0 | 0 |
invalid-return-type |
0 | 0 | 1 |
unsupported-operator |
0 | 1 | 0 |
unused-ignore-comment |
0 | 1 | 0 |
| Total | 21 | 8 | 25 |
ibraheemdev
force-pushed
the
ibraheem/declared-type-context
branch
from
0fbaae1 to
9b81c82
Compare
|
This is more complicated for class attributes as may need to infer the RHS multiple times during attribute resolution. This is similar to what we do for function overloads, but the difference is that the RHS of an assignment statement is a standalone expression, so I'm not exactly sure how to approach this. We may be able to collect the possible type contexts beforehand and infer the intersection as the single type for the expression, but this needs more work. I'm going to cut down this PR to just handle simple assignment statements, and implement class attributes in a followup PR. |
|
Took a look at some of the ecosystem hits here, just documenting what I found:
IIRC our approach to type context is generally just to union it in in the specialization builder? It seems like this approach may need some refinement in cases where the type context is a union, so we avoid unioning in any elements of the context union that are definitely not part of the type of the expression we are inferring? |
|
I guess the issue isn't really new to this PR, it already exists: https://play.ty.dev/6d7b5e71-848a-4091-90ea-b56f548a5eff I think in that case it's going to be important that we can at least narrow away the |
|
#20528 adds a |
ibraheemdev
force-pushed
the
ibraheem/declared-type-context
branch
from
9b81c82 to
29da7aa
Compare
Discussed this a bit with @ibraheemdev just now, and I think a rewording of this that I strongly agree with is that we should reveal the same types for def id[T](x: T) -> T:
return x
def _(i: int):
x: int | None = id(i)
y: int | None = i
reveal_type(x)
reveal_type(y) # revealed: int |
ibraheemdev
changed the base branch from
main
to
ibraheem/generic-call-inference
ibraheemdev
force-pushed
the
ibraheem/generic-call-inference
branch
4 times, most recently
from
af6dad0 to
5c817d8
Compare
ibraheemdev
force-pushed
the
ibraheem/declared-type-context
branch
from
29da7aa to
3bf25d1
Compare
dcreager
reviewed
Oct 16, 2025
There was a problem hiding this comment.
Code looks good. Have you looked through the remaining ecosystem report results? (Also, does the ecosystem report show the diff between this PR and #20875, or main? If the latter we should want until that lands and get a fresh report to look at.)
Comment on lines
-1335
to
+1357
| fn add_binding(&mut self, node: AnyNodeRef, binding: Definition<'db>, ty: Type<'db>) { | ||
| /// Add a binding for the given definition. | ||
| /// | ||
| /// Returns the result of the `infer_value_ty` closure, which is called with the declared type | ||
| /// as type context. | ||
| fn add_binding( | ||
| &mut self, | ||
| node: AnyNodeRef, | ||
| binding: Definition<'db>, | ||
| infer_value_ty: impl FnOnce(&mut Self, TypeContext<'db>) -> Type<'db>, | ||
| ) -> Type<'db> { |
There was a problem hiding this comment.
nit: Most of the callers below don't look to be using the type context, and provide a closure that just immediately returns the previous ty parameter. You can reduce the churn on the diff by keeping add_binding with its existing signature, and calling this new variant e.g. add_binding_with_type_context.
fn add_binding(&mut self, node: AnyNodeRef, binding: Definition<'db>, ty: Type<'db>) {
self.add_binding_with_context(node, binding, |_, _| ty);
}There was a problem hiding this comment.
This has come up a few times, and I think I would prefer to keep the signature as is for now, because I think it's useful to know when there is a potential type context available that is being ignored, and in fact there are a couple of cases where we probably could be using the type context, but I've held off for now.
|
The diff is between this PR and #20875. The ecosystem report looks good, mostly positive changes and a couple regressions due to other known limitations (i.e. astral-sh/ty#1248). |
ibraheemdev
force-pushed
the
ibraheem/generic-call-inference
branch
from
e9e278f to
d806594
Compare
ibraheemdev
added a commit
that referenced
this pull request
Oct 16, 2025
## Summary Ignore the type context when specializing a generic call if it leads to an unnecessarily wide return type. For example, [the example mentioned here](#20796 (comment)) works as expected after this change: ```py def id[T](x: T) -> T: return x def _(i: int): x: int | None = id(i) y: int | None = i reveal_type(x) # revealed: int reveal_type(y) # revealed: int ``` I also added extended our usage of `filter_disjoint_elements` to tuple and typed-dict inference, which resolves astral-sh/ty#1266.
ibraheemdev
force-pushed
the
ibraheem/declared-type-context
branch
from
3bf25d1 to
9f3fa9f
Compare
dcreager
approved these changes
Oct 16, 2025
dcreager
added a commit
that referenced
this pull request
Oct 17, 2025
* main: [ty] Prefer declared type for invariant collection literals (#20927) [ty] Don't track inferability via different `Type` variants (#20677) [ty] Use declared variable types as bidirectional type context (#20796) [ty] Avoid unnecessarily widening generic specializations (#20875) [ty] Support dataclass-transform `field_specifiers` (#20888) Bump 0.14.1 (#20925) Standardize syntax error construction (#20903) [`pydoclint`] Implement `docstring-extraneous-parameter` (`DOC102`) (#20376) [ty] Fix panic 'missing root' when handling completion request (#20917) [ty] Run file watching tests serial when using nextest (#20918) [ty] Add version hint for failed stdlib attribute accesses (#20909) More CI improvements (#20920) [ty] Check typeshed VERSIONS for parent modules when reporting failed stdlib imports (#20908)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Use the declared type of variables as type context for the RHS of assignment expressions, e.g.,