Add Client Middleware Support

[bdraco]

Summary

This PR introduces middleware support for the aiohttp client, allowing users to intercept and modify requests and responses. This brings the client API closer to parity with the server API and enables powerful use cases like authentication, logging, retries, and request/response modification.

Key Changes

Core Implementation

• Added aiohttp.client_middlewares module with:

  • ClientMiddlewareType type alias for middleware functions
  • ClientHandlerType type alias for request handlers
  • build_client_middlewares() function to chain middlewares efficiently

• Modified ClientSession to accept middlewares parameter:

  • Supports both session-level and request-level middleware
  • Added _middlewares attribute to store session middlewares
  • Modified request handling to apply middleware chain

• Added session attribute to ClientRequest:

  • Allows middleware to access the parent session
  • Enables middleware to make additional requests (useful for OAuth token refresh, retries, etc.)

Documentation

• Comprehensive documentation in client_advanced.rst:
  • Basic middleware structure and types
  • How to apply middleware at session and request level
  • Common middleware patterns (logging, authentication, retries)
  • Middleware execution order and chaining
  • Best practices and caveats

Tests

• Extensive test coverage in test_client_middleware.py (~1100 lines):
  • Basic middleware functionality
  • Multiple middleware chaining
  • Order of execution
  • Request/response modification
  • Error handling
  • Retry patterns
  • Recursion prevention
  • Integration with existing client features

Features Enabled

1. Authentication Middleware: Automatically add authentication headers or refresh tokens
2. Logging Middleware: Log all requests and responses
3. Retry Middleware: Implement custom retry logic with backoff
4. Request/Response Modification: Modify headers, URLs, or response data
5. Circuit Breaker Pattern: Implement circuit breakers for failing services
6. Rate Limiting: Implement client-side rate limiting

Example Usage

# Basic middleware
async def logging_middleware(request, handler):
    print(f"Request: {request.method} {request.url}")
    response = await handler(request)
    print(f"Response: {response.status}")
    return response

# Apply to session
async with ClientSession(middlewares=(logging_middleware,)) as session:
    resp = await session.get('http://example.com')

# Apply to specific request
async with ClientSession() as session:
    resp = await session.get('http://example.com', middlewares=(logging_middleware,))

Technical Details

• Middleware execution is optimized for minimal overhead
• Single middleware case has a dedicated fast path
• Middleware chain is built without using partial or update_wrapper to reduce overhead
• No caching of middleware chains to avoid holding references to stateful middleware

Breaking Changes

None. This is a backwards-compatible addition.

Related Issues

• Fixes Support dynamically updating authentication headers #6915 - Request for client middleware support
• Fixes Implementing auth handlers with aiohttp.client #434 - Add middleware for client requests
• Addresses support for digest authentication #4939 - Client middleware patterns for authentication and retries

[codspeed-hq]

CodSpeed Performance Report

Merging #9732 will not alter performance

_{Comparing client_middleware (ab1aac2) with master (e5d1240)}

Summary

✅ 60 untouched benchmarks

[Dreamsorcerer]

Also #6915. There are some examples in there of different use cases which should work when this is complete.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.74%. Comparing base (e5d1240) to head (ab1aac2).
⚠️ Report is 516 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9732      +/-   ##
==========================================
+ Coverage   98.71%   98.74%   +0.02%     
==========================================
  Files         125      127       +2     
  Lines       37894    38520     +626     
  Branches     2094     2123      +29     
==========================================
+ Hits        37407    38036     +629     
+ Misses        338      336       -2     
+ Partials      149      148       -1     

Flag	Coverage Δ
CI-GHA	98.62% <100.00%> (+0.03%)	⬆️
OS-Linux	98.33% <100.00%> (+0.03%)	⬆️
OS-Windows	96.49% <100.00%> (+0.07%)	⬆️
OS-macOS	97.46% <100.00%> (+0.04%)	⬆️
Py-3.10.11	97.35% <100.00%> (+0.04%)	⬆️
Py-3.10.17	97.86% <100.00%> (+<0.01%)	⬆️
Py-3.11.12	97.98% <100.00%> (+0.03%)	⬆️
Py-3.11.9	97.44% <100.00%> (+0.04%)	⬆️
Py-3.12.10	98.40% <100.00%> (+0.03%)	⬆️
Py-3.13.3	98.38% <100.00%> (+0.03%)	⬆️
Py-3.9.13	97.23% <100.00%> (+0.06%)	⬆️
Py-3.9.22	97.76% <100.00%> (+0.03%)	⬆️
Py-pypy7.3.16	93.65% <100.00%> (+0.14%)	⬆️
VM-macos	97.46% <100.00%> (+0.04%)	⬆️
VM-ubuntu	98.33% <100.00%> (+0.03%)	⬆️
VM-windows	96.49% <100.00%> (+0.07%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bdraco]

Since we're generally happy with the design now (as discussed on Matrix), I'll go ahead and merge this to unblock the follow-up work. Any further adjustments can be handled in a separate PR.

[patchback]

Backport to 3.12: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 1e911ea on top of patchback/backports/3.12/1e911eaf23091b42cc374547ff9bf692debf9720/pr-9732

Backporting merged PR #9732 into master

1. Ensure you have a local repo clone of your fork. Unless you cloned it
   from the upstream, this would be your origin remote.
2. Make sure you have an upstream repo added as a remote too. In these
   instructions you'll refer to it by the name upstream. If you don't
   have it, here's how you can add it:

   $ git remote add upstream https://github.com/aio-libs/aiohttp.git

3. Ensure you have the latest copy of upstream and prepare a branch
   that will hold the backported code:

   $ git fetch upstream
   $ git checkout -b patchback/backports/3.12/1e911eaf23091b42cc374547ff9bf692debf9720/pr-9732 upstream/3.12

4. Now, cherry-pick PR Add Client Middleware Support #9732 contents into that branch:

   $ git cherry-pick -x 1e911eaf23091b42cc374547ff9bf692debf9720

   If it'll yell at you with something like fatal: Commit 1e911eaf23091b42cc374547ff9bf692debf9720 is a merge but no -m option was given., add -m 1 as follows instead:

   $ git cherry-pick -m1 -x 1e911eaf23091b42cc374547ff9bf692debf9720

5. At this point, you'll probably encounter some merge conflicts. You must
   resolve them in to preserve the patch from PR Add Client Middleware Support #9732 as close to the
   original as possible.
6. Push this branch to your fork on GitHub:

   $ git push origin patchback/backports/3.12/1e911eaf23091b42cc374547ff9bf692debf9720/pr-9732

7. Create a PR, ensure that the CI is green. If it's not — update it so that
   the tests and any other checks pass. This is it!
   Now relax and wait for the maintainers to process your pull request
   when they have some cycles to do reviews. Don't worry — they'll tell you if
   any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

[bdraco]

Didn't think I would be that lucky that it wouldn't conflict 😄

[bdraco]

Thanks for the review. I'll clean this up between sprint work.