spec: Rewrite TypedDict spec (#2072)

* spec: Rewrite TypedDict spec

This is an edit of the TypedDict spec for clarity and flow. My goal was to
unify the pieces of the spec that derive from the various PEPs into a coherent
whole. I removed excessive examples and motivations: the spec should specify,
not justify. The length of the spec chapter is reduced by more than half.

This change is on top of #2068 (adding PEP 728).

The general approach I took is to first define the kinds of TypedDicts that
can exist, then explain the syntax for defining TypedDicts, then discuss
other aspects of TypedDict types.

I introduce some new terminology around PEP 728 to make it easier to talk
about the different kinds of TypedDict. TypedDicts are defined to have a
property called openness, which can have three states:
- Open: all TypedDicts prior to PEP 728
- Closed: no extra keys are allowed (closed=True)
- With extra items: extra_items=... from PEP 728

I retained existing text where it made sense but also wrote some from
scratch.

* Apply suggestions from code review

Co-authored-by: Joren Hammudoglu <[email protected]>

* feedback

* Carl feedback

* more clarity

---------

Co-authored-by: Joren Hammudoglu <[email protected]>

Author: JelleZijlstra

docs/spec/callables.rst

@@ -180,24 +180,61 @@ generated. For example::
                                                             # so **kwargs can contain
                                                             # a "name" keyword.
 
-Required and non-required keys
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-By default all keys in a ``TypedDict`` are required. This behavior can be
-overridden by setting the dictionary's ``total`` parameter as ``False``.
-Moreover, :pep:`655` introduced new type qualifiers - ``typing.Required`` and
-``typing.NotRequired`` - that enable specifying whether a particular key is
-required or not::
-
-    class Movie(TypedDict):
-        title: str
-        year: NotRequired[int]
+Required and non-required items
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
+Items in a TypedDict may be either :term:`required` or :term:`non-required`.
 When using a ``TypedDict`` to type ``**kwargs`` all of the required and
 non-required keys should correspond to required and non-required function
-keyword parameters. Therefore, if a required key is not supported by the
+keyword parameters. Therefore, if a required key is not provided by the
 caller, then an error must be reported by type checkers.
 
+Read-only items
+^^^^^^^^^^^^^^^
+
+TypedDict items may also be :term:`read-only`. Marking one or more of the items of a TypedDict
+used to type ``**kwargs`` as read-only will have no effect on the type signature of the method.
+However, it *will* prevent the item from being modified in the body of the function::
+
+    class Args(TypedDict):
+        key1: int
+        key2: str
+
+    class ReadOnlyArgs(TypedDict):
+        key1: ReadOnly[int]
+        key2: ReadOnly[str]
+
+    class Function(Protocol):
+        def __call__(self, **kwargs: Unpack[Args]) -> None: ...
+
+    def impl(**kwargs: Unpack[ReadOnlyArgs]) -> None:
+        kwargs["key1"] = 3  # Type check error: key1 is readonly
+
+    fn: Function = impl  # Accepted by type checker: function signatures are identical
+
+Extra items
+^^^^^^^^^^^
+
+If the TypedDict used for annotating ``**kwargs`` is defined to allow
+:term:`extra items`, arbitrary additional keyword arguments of the right
+type may be passed to the function::
+
+    class MovieNoExtra(TypedDict):
+        name: str
+
+    class MovieExtra(TypedDict, extra_items=int):
+        name: str
+
+    def f(**kwargs: Unpack[MovieNoExtra]) -> None: ...
+    def g(**kwargs: Unpack[MovieExtra]) -> None: ...
+
+    # Should be equivalent to:
+    def f(*, name: str) -> None: ...
+    def g(*, name: str, **kwargs: int) -> None: ...
+
+    f(name="No Country for Old Men", year=2007) # Not OK. Unrecognized item
+    g(name="No Country for Old Men", year=2007) # OK
+
 Assignment
 ^^^^^^^^^^
 

docs/spec/concepts.rst

@@ -69,7 +69,7 @@ attributes and/or methods.
 
 If an object ``v`` is a member of the set of objects denoted by a fully static
 type ``T``, we can say that ``v`` is a "member of" the type ``T``, or ``v``
-"inhabits" ``T``.
+":term:`inhabits <inhabit>`" ``T``.
 
 Gradual types
 ~~~~~~~~~~~~~
@@ -298,9 +298,9 @@ visualize this analogy in the following table:
    * - ``B`` is :term:`equivalent` to ``A``
      - ``B`` is :term:`consistent` with ``A``
 
-We can also define an **equivalence** relation on gradual types: the gradual 
-types ``A`` and ``B`` are equivalent (that is, the same gradual type, not 
-merely consistent with one another) if and only if all materializations of 
+We can also define an **equivalence** relation on gradual types: the gradual
+types ``A`` and ``B`` are equivalent (that is, the same gradual type, not
+merely consistent with one another) if and only if all materializations of
 ``A`` are also materializations of ``B``, and all materializations of ``B``
 are also materializations of ``A``.
 

docs/spec/glossary.rst

@@ -27,6 +27,13 @@ This section defines a few terms that may be used elsewhere in the specification
       ``B``, respectively, such that ``B'`` is a subtype of ``A'``. See
       :ref:`type-system-concepts`.
 
+   closed
+      A :ref:`TypedDict <typeddict>` type is closed if it may not contain any
+      additional :term:`items <item>` beyond those specified in the TypedDict definition.
+      A closed TypedDict can be created using the ``closed=True`` argument to
+      :py:func:`typing.TypedDict`, or equivalently by setting ``extra_items=Never``.
+      Compare :term:`extra items` and :term:`open`.
+
    consistent
       Two :term:`fully static types <fully static type>` are "consistent with"
       each other if they are :term:`equivalent`. Two gradual types are
@@ -52,6 +59,14 @@ This section defines a few terms that may be used elsewhere in the specification
       also materializations of ``B``, and all materializations of ``B`` are
       also materializations of ``A``.
 
+   extra items
+      A :ref:`TypedDict <typeddict>` type with extra items may contain arbitrary
+      additional key-value pairs beyond those specified in the TypedDict definition, but those
+      values must be of the type specified by the ``extra_items=`` argument to the definition.
+      A TypedDict with extra items can be created using the ``extra_items=``
+      argument to :py:func:`typing.TypedDict`. Extra items may or may not be
+      :term:`read-only`. Compare :term:`closed` and :term:`open`.
+
    fully static type
       A type is "fully static" if it does not contain any :term:`gradual form`.
       A fully static type represents a set of possible runtime values. Fully
@@ -79,11 +94,21 @@ This section defines a few terms that may be used elsewhere in the specification
       They can be :term:`materialized <materialize>` to a more static, or fully static,
       type. See :ref:`type-system-concepts`.
 
+   inhabit
+      A value is said to inhabit a type if it is a member of the set of values
+      represented by that type. For example, the value ``42`` inhabits the type
+      ``int``, and the value ``"hello"`` inhabits the type ``str``.
+
    inline
       Inline type annotations are annotations that are included in the
       runtime code using :pep:`526` and
       :pep:`3107` syntax (the filename ends in ``.py``).
 
+   item
+      In the context of a :ref:`TypedDict <typeddict>`, an item consists of a name
+      (the dictionary key) and a type (representing the type that values corresponding to the key must have).
+      Items may be :term:`required` or :term:`non-required`, and may be :term:`read-only` or writable.
+
    materialize
       A :term:`gradual type` can be materialized to a more static type
       (possibly a :term:`fully static type`) by replacing :ref:`Any` with any
@@ -110,12 +135,41 @@ This section defines a few terms that may be used elsewhere in the specification
       ``__class__`` is that type, or any of its subclasses, transitively. In
       contrast, see :term:`structural` types.
 
+   non-required
+      If an :term:`item` in a :ref:`TypedDict <typeddict>` is non-required, it may or
+      may not be present on an object of that TypedDict type, but if it is present
+      it must be of the type specified by the TypedDict definition.
+      Items can be marked as non-required using the :py:data:`typing.NotRequired` qualifier
+      or the ``total=False`` argument to :py:func:`typing.TypedDict`. Compare :term:`required`.
+
+   open
+      A :ref:`TypedDict <typeddict>` type is open if it may contain arbitrary
+      additional :term:`items <item>` beyond those specified in the TypedDict definition.
+      This is the default behavior for TypedDicts that do not use the ``closed=True``
+      or ``extra_items=`` arguments to :py:func:`typing.TypedDict`.
+      Open TypedDicts behave similarly to TypedDicts with :term:`extra items` of type
+      ``ReadOnly[object]``, but differ in some behaviors; see the TypedDict specification
+      chapter for details.
+      Compare :term:`extra items` and :term:`closed`.
+
    package
       A directory or directories that namespace Python modules.
       (Note the distinction between packages and :term:`distributions <distribution>`.
       While most distributions are named after the one package they install, some
       distributions install multiple packages.)
 
+   read-only
+      A read-only :term:`item` in a :ref:`TypedDict <typeddict>` may not be modified.
+      Attempts to delete or assign to that item
+      should be reported as type errors by a type checker. Read-only items are created
+      using the :py:data:`typing.ReadOnly` qualifier.
+
+   required
+      If an :term:`item` in a :ref:`TypedDict <typeddict>` is required, it must be present
+      in any object of that TypedDict type. Items are
+      required by default, but items can also be explicitly marked as required using
+      the :py:data:`typing.Required` qualifier. Compare :term:`non-required`.
+
    special form
       A special form is an object that has a special meaning within the type system,
       comparable to a keyword in the language grammar. Examples include ``Any``,