Incorporated EP2025 sprint feedback and added a new section (#955)

Also fixed a number of broken documentation links.

Author: agronholm

README.rst

@@ -12,13 +12,16 @@
   :alt: Gitter chat
 
 AnyIO is an asynchronous networking and concurrency library that works on top of either asyncio_ or
-trio_. It implements trio-like `structured concurrency`_ (SC) on top of asyncio and works in harmony
-with the native SC of trio itself.
+Trio_. It implements Trio-like `structured concurrency`_ (SC) on top of asyncio and works in harmony
+with the native SC of Trio itself.
 
 Applications and libraries written against AnyIO's API will run unmodified on either asyncio_ or
-trio_. AnyIO can also be adopted into a library or application incrementally – bit by bit, no full
+Trio_. AnyIO can also be adopted into a library or application incrementally – bit by bit, no full
 refactoring necessary. It will blend in with the native libraries of your chosen backend.
 
+To find out why you might want to use AnyIO's APIs instead of asyncio's, you can read about it
+`here <https://anyio.readthedocs.io/en/stable/why.html>`_.
+
 Documentation
 -------------
 
@@ -49,7 +52,7 @@ AnyIO also comes with its own pytest_ plugin which also supports asynchronous fi
 It even works with the popular Hypothesis_ library.
 
 .. _asyncio: https://docs.python.org/3/library/asyncio.html
-.. _trio: https://github.com/python-trio/trio
+.. _Trio: https://github.com/python-trio/trio
 .. _structured concurrency: https://en.wikipedia.org/wiki/Structured_concurrency
 .. _nurseries: https://trio.readthedocs.io/en/stable/reference-core.html#nurseries-and-spawning
 .. _Happy eyeballs: https://en.wikipedia.org/wiki/Happy_Eyeballs

docs/api.rst

@@ -131,6 +131,7 @@ Streams and stream wrappers
 .. autodata:: anyio.abc.AnyByteStreamConnectable
 
 .. autoclass:: anyio.streams.buffered.BufferedByteReceiveStream
+.. autoclass:: anyio.streams.buffered.BufferedByteStream
 .. autoclass:: anyio.streams.file.FileStreamAttribute
 .. autoclass:: anyio.streams.file.FileReadStream
 .. autoclass:: anyio.streams.file.FileWriteStream
@@ -172,6 +173,8 @@ Sockets and networking
 .. autoclass:: anyio.abc.UDPSocket()
 .. autoclass:: anyio.abc.ConnectedUDPSocket()
 .. autoclass:: anyio.abc.UNIXSocketStream()
+.. autoclass:: anyio.abc.UNIXDatagramSocket()
+.. autoclass:: anyio.abc.ConnectedUNIXDatagramSocket()
 .. autoclass:: anyio.TCPConnectable
 .. autoclass:: anyio.UNIXConnectable
 

docs/basics.rst

@@ -34,10 +34,11 @@ The simplest possible AnyIO program looks like this::
 
     run(main)
 
-This will run the program above on the default backend (asyncio). To run it on another
-supported backend, say Trio_, you can use the ``backend`` argument, like so::
+This will run the program above on the default backend (asyncio). To explicitly specify
+which backend to run on, you can use the ``backend`` argument, like so::
 
     run(main, backend='trio')
+    run(main, backend='asyncio')
 
 But AnyIO code is not required to be run via :func:`run`. You can just as well use the
 native ``run()`` function of the backend library::
@@ -63,6 +64,14 @@ native ``run()`` function of the backend library::
 Backend specific options
 ------------------------
 
+Any options exclusive to a specific backend can be passed as keyword arguments to
+:func:`run`::
+
+    run(main, backend="asyncio", debug=True)
+    run(main, backend="trio", restrict_keyboard_interrupt_to_checkpoints=True)
+
+Here is the list of supported options for each backend:
+
 **Asyncio**:
 
 * options covered in the documentation of :class:`asyncio.Runner`
@@ -95,5 +104,7 @@ asynchronous framework of your choice. There are a few rules to keep in mind how
 * Threads spawned outside of AnyIO cannot use :func:`.from_thread.run` to call
   asynchronous code
 
+.. seealso:: :ref:`asyncio cancellation`
+
 .. _virtualenv: https://docs.python-guide.org/dev/virtualenvs/
 .. _Trio: https://github.com/python-trio/trio

docs/cancellation.rst

@@ -15,7 +15,34 @@ If the task is just starting, it will run until it first tries to run an operati
 requiring waiting, such as :func:`~sleep`.
 
 A task group contains its own cancel scope. The entire task group can be cancelled by
-cancelling this scope.
+cancelling this scope::
+
+    from anyio import create_task_group, get_cancelled_exc_class, sleep, run
+
+
+    async def waiter(index: int):
+        try:
+            await sleep(1)
+        except get_cancelled_exc_class():
+            print(f"Waiter {index} cancelled")
+            raise
+
+
+    async def taskfunc():
+        async with create_task_group() as tg:
+            # Start a couple tasks and wait until they are blocked
+            tg.start_soon(waiter, 1)
+            tg.start_soon(waiter, 2)
+            await sleep(0.1)
+
+            # Cancel the scope and exit the task group
+            tg.cancel_scope.cancel()
+
+    run(taskfunc)
+
+    # Output:
+    # Waiter 1 cancelled
+    # Waiter 2 cancelled
 
 .. _Trio: https://trio.readthedocs.io/en/latest/reference-core.html
    #cancellation-and-timeouts
@@ -126,6 +153,7 @@ itself is being cancelled. Shielding a cancel scope is often best combined with
 
     run(main)
 
+.. _finalization:
 
 Finalization
 ------------

docs/index.rst

@@ -26,6 +26,7 @@ The manual
    testing
    api
    migration
+   why
    faq
    support
    contributing

docs/networking.rst

@@ -255,12 +255,14 @@ AnyIO stream or socket listener. For that, various class methods exist:
 * :meth:`.abc.UNIXDatagramSocket.from_socket`
 * :meth:`.abc.ConnectedUNIXDatagramSocket.from_socket`
 
+.. _connectables:
+
 Abstracting remote connections using Connectables
 -------------------------------------------------
 
 AnyIO offers a hierarchy of classes implementing either the
-:class:`ObjectStreamConnectable` or :class:`ByteStreamConnectable` interfaces which
-lets developers abstract out the connection mechanism for network clients.
+:class:`.abc.ObjectStreamConnectable` or :class:`.abc.ByteStreamConnectable` interfaces
+which lets developers abstract out the connection mechanism for network clients.
 For example, you could create a network client class like this::
 
     from os import PathLike
@@ -289,7 +291,7 @@ For example, you could create a network client class like this::
 
 Here's a dissection of the type annotation for ``connectable``:
 
-* :class:`ByteStreamConnectable`: allows for any arbitrary bytestream connectable
+* :class:`.abc.ByteStreamConnectable`: allows for any arbitrary bytestream connectable
 * ``tuple[str, int]``: TCP host/port
 * ``str | bytes | PathLike[str]``: file system path to a UNIX socket
 

docs/subinterpreters.rst

@@ -24,7 +24,7 @@ Running functions in a worker interpreter makes sense when:
 If the code you're trying to run only does blocking network I/O, or file I/O, then
 you're better off using :doc:`worker thread <threads>` instead.
 
-This is done by using :func:`.interpreter.run_sync`::
+This is done by using :func:`.to_interpreter.run_sync`::
 
     import time
 

docs/synchronization.rst

@@ -13,9 +13,9 @@ notifying other tasks and guarding access to shared resources.
 Events
 ------
 
-Events are used to notify tasks that something they've been waiting to happen has
-happened. An event object can have multiple listeners and they are all notified when the
-event is triggered.
+Events (:class:`Event`) are used to notify tasks that something they've been waiting to
+happen has happened. An event object can have multiple listeners and they are all
+notified when the event is triggered.
 
 Example::
 
@@ -35,17 +35,20 @@ Example::
 
     run(main)
 
+    # Output:
+    # Received notification!
+
 .. note:: Unlike standard library Events, AnyIO events cannot be reused, and must be
    replaced instead. This practice prevents a class of race conditions, and matches the
    semantics of the Trio library.
 
 Semaphores
 ----------
 
-Semaphores are used for limiting access to a shared resource. A semaphore starts with a
-maximum value, which is decremented each time the semaphore is acquired by a task and
-incremented when it is released. If the value drops to zero, any attempt to acquire the
-semaphore will block until another task frees it.
+Semaphores (:class:`Semaphore`) are used for limiting access to a shared resource. A
+semaphore starts with a maximum value, which is decremented each time the semaphore is
+acquired by a task and incremented when it is released. If the value drops to zero, any
+attempt to acquire the semaphore will block until another task frees it.
 
 Example::
 
@@ -54,7 +57,7 @@ Example::
 
     async def use_resource(tasknum, semaphore):
         async with semaphore:
-            print('Task number', tasknum, 'is now working with the shared resource')
+            print(f"Task number {tasknum} is now working with the shared resource")
             await sleep(1)
 
 
@@ -66,6 +69,18 @@ Example::
 
     run(main)
 
+    # Output:
+    # Task number 0 is now working with the shared resource
+    # Task number 1 is now working with the shared resource
+    # Task number 2 is now working with the shared resource
+    # Task number 3 is now working with the shared resource
+    # Task number 4 is now working with the shared resource
+    # Task number 5 is now working with the shared resource
+    # Task number 6 is now working with the shared resource
+    # Task number 7 is now working with the shared resource
+    # Task number 8 is now working with the shared resource
+    # Task number 9 is now working with the shared resource
+
 .. tip:: If the performance of semaphores is critical for you, you could pass
    ``fast_acquire=True`` to :class:`Semaphore`. This has the effect of skipping the
    :func:`~.lowlevel.cancel_shielded_checkpoint` call in :meth:`Semaphore.acquire` if
@@ -76,9 +91,9 @@ Example::
 Locks
 -----
 
-Locks are used to guard shared resources to ensure sole access to a single task at once.
-They function much like semaphores with a maximum value of 1, except that only the task
-that acquired the lock is allowed to release it.
+Locks (:class:`Lock`) are used to guard shared resources to ensure sole access to a
+single task at once. They function much like semaphores with a maximum value of 1,
+except that only the task that acquired the lock is allowed to release it.
 
 Example::
 
@@ -99,6 +114,12 @@ Example::
 
     run(main)
 
+    # Output:
+    # Task number 0 is now working with the shared resource
+    # Task number 1 is now working with the shared resource
+    # Task number 2 is now working with the shared resource
+    # Task number 3 is now working with the shared resource
+
 .. tip:: If the performance of locks is critical for you, you could pass
    ``fast_acquire=True`` to :class:`Lock`. This has the effect of skipping the
    :func:`~.lowlevel.cancel_shielded_checkpoint` call in :meth:`Lock.acquire` if there
@@ -148,12 +169,31 @@ Example::
 
     run(main)
 
+    # Output:
+    # Woke up task number 0
+    # Woke up task number 1
+    # Woke up task number 2
+    # Woke up task number 3
+    # Woke up task number 4
+    # Woke up task number 5
+
+.. _capacity-limiters:
+
 Capacity limiters
 -----------------
 
-Capacity limiters are like semaphores except that a single borrower (the current task by
-default) can only hold a single token at a time. It is also possible to borrow a token
-on behalf of any arbitrary object, so long as that object is hashable.
+Capacity limiters (:class:`CapacityLimiter`) are like semaphores except that a single
+borrower (the current task by default) can only hold a single token at a time. It is
+also possible to borrow a token on behalf of any arbitrary object, so long as that object
+is hashable.
+
+It is recommended to use capacity limiters instead of semaphores unless you intend to
+allow a task to acquire multiple tokens from the same object. AnyIO uses capacity
+limiters to limit the number of threads spawned.
+
+The number of total tokens available for tasks to acquire can be adjusted by assigning
+the desired value to the ``total_tokens`` property. If the value is higher than the
+previous one, it will automatically wake up the appropriate number of waiting tasks.
 
 Example::
 
@@ -162,7 +202,7 @@ Example::
 
     async def use_resource(tasknum, limiter):
         async with limiter:
-            print('Task number', tasknum, 'is now working with the shared resource')
+            print(f"Task number {tasknum} is now working with the shared resource")
             await sleep(1)
 
 
@@ -174,8 +214,17 @@ Example::
 
     run(main)
 
-You can adjust the total number of tokens by setting a different value on the limiter's
-``total_tokens`` property.
+    # Output:
+    # Task number 0 is now working with the shared resource
+    # Task number 1 is now working with the shared resource
+    # Task number 2 is now working with the shared resource
+    # Task number 3 is now working with the shared resource
+    # Task number 4 is now working with the shared resource
+    # Task number 5 is now working with the shared resource
+    # Task number 6 is now working with the shared resource
+    # Task number 7 is now working with the shared resource
+    # Task number 8 is now working with the shared resource
+    # Task number 9 is now working with the shared resource
 
 Resource guards
 ---------------

docs/tasks.rst

@@ -41,6 +41,8 @@ Here's a demonstration::
 .. _Trio: https://trio.readthedocs.io/en/latest/reference-core.html
    #tasks-let-you-do-multiple-things-at-once
 
+.. _start_initialize:
+
 Starting and initializing tasks
 -------------------------------
 
@@ -145,9 +147,22 @@ function from the exceptiongroup_ package::
 
 If you need to set local variables in the handlers, declare them as ``nonlocal``::
 
-    def handle_valueerror(exc):
-        nonlocal somevariable
-        somevariable = 'whatever'
+    async def yourfunc():
+        somevariable: str | None = None
+
+        def handle_valueerror(exc):
+            nonlocal somevariable
+            somevariable = 'whatever'
+
+        with catch({
+            ValueError: handle_valueerror,
+            KeyError: handle_keyerror
+        }):
+            async with create_task_group() as tg:
+                tg.start_soon(some_task)
+                tg.start_soon(another_task)
+
+        print(f"{somevariable=}")
 
 .. _exceptiongroup: https://pypi.org/project/exceptiongroup/
 

docs/testing.rst

@@ -2,7 +2,8 @@ Testing with AnyIO
 ==================
 
 AnyIO provides built-in support for testing your library or application in the form of a
-pytest_ plugin.
+pytest_ plugin. This plugin is part of the AnyIO distribution, so nothing extra needs to
+be installed to use it.
 
 .. _pytest: https://docs.pytest.org/en/latest/
 
@@ -146,9 +147,9 @@ Built-in utility fixtures
 Some useful pytest fixtures are provided to make testing network services easier:
 
 * ``free_tcp_port_factory``: session scoped fixture returning a callable
-  (:class:`~pytest_plugin.FreePortFactory`) that generates unused TCP port numbers
+  (:class:`~.pytest_plugin.FreePortFactory`) that generates unused TCP port numbers
 * ``free_udp_port_factory``: session scoped fixture returning a callable
-  (:class:`~pytest_plugin.FreePortFactory`) that generates unused UDP port numbers
+  (:class:`~.pytest_plugin.FreePortFactory`) that generates unused UDP port numbers
 * ``free_tcp_port``: function level fixture that invokes the ``free_tcp_port_factory``
   fixture to generate a free TCP port number
 * ``free_udp_port``: function level fixture that invokes the ``free_udp_port_factory``