[PR #11054/e2eb1959 backport][3.12] Fix CookieJar memory leak in filter_cookies() #11068
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a backport of PR #11054 as merged into master (e2eb195).
Summary
This PR fixes a memory leak in
CookieJar.filter_cookies()
that was causing unbounded memory growth when making requests to different URL paths. The issue was introduced in version 3.10.0 and could cause memory usage to grow from ~100MB to ~5GB over weeks of operation.The Problem
The
filter_cookies()
method was inadvertently creating empty cookie entries for every domain-path combination it checked. This happened because:self._cookies[p]
for each combinationself._cookies
is adefaultdict(SimpleCookie)
, accessing a non-existent key creates an emptySimpleCookie
objectThe Fix
Added a simple check before accessing the dictionary:
This prevents the creation of empty cookie entries while maintaining all existing functionality.
Testing
test_filter_cookies_does_not_leak_memory()
that verifies the internal storage doesn't grow when filtering cookies for different pathsImpact