Skip to content

Conversation

@flamingbear
Copy link
Member

@flamingbear flamingbear commented Nov 20, 2025

Jira Issue ID

None

Description

Removes backoff_jitter from the Retry() call.

This keyword was added to urllib3 in v2.0.0. Some users are running into issues, unable to initialize a Client with urllib3 < 2.

I don't want to dig around to handle multiple library versions. I'm proposing to just nuke this option.

Error case

❯ python
Python 3.12.12 | packaged by conda-forge | (main, Oct 22 2025, 23:34:53) [Clang 19.1.7 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib3
>>> urllib3.__version__
'1.26.20'
>>> import harmony
>>> harmony.__version__
'1.3.2'
>>> harmony.Client()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/savoie/.pyenv/versions/miniforge3-24.3.0-0/envs/nsidc-tutorials/lib/python3.12/site-packages/harmony/client.py", line 160, in __init__
    validate_auth(self.config, self._session())
                               ^^^^^^^^^^^^^^^
  File "/Users/savoie/.pyenv/versions/miniforge3-24.3.0-0/envs/nsidc-tutorials/lib/python3.12/site-packages/harmony/client.py", line 170, in _session
    retry_strategy = Retry(
                     ^^^^^^
TypeError: Retry.__init__() got an unexpected keyword argument 'backoff_jitter'

Success case (after installing this branch's built package)

Python 3.12.12 | packaged by conda-forge | (main, Oct 22 2025, 23:34:53) [Clang 19.1.7 ] on darwin
>>> import urllib3
>>> urllib3.__version__
'1.26.20'
❯ pip install ../data-services/harmony-py/dist/harmony_py-1.3.2+1.gde0d5a8-py3-none-any.whl
Processing /Users/savoie/projects/data-services/harmony-py/dist/harmony_py-1.3.2+1.gde0d5a8-py3-none-any.whl
(snip)
Successfully installed harmony-py-1.3.2+1.gde0d5a8 python-dotenv-1.1.1
❯ python
Python 3.12.12 | packaged by conda-forge | (main, Oct 22 2025, 23:34:53) [Clang 19.1.7 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import harmony
>>> harmony.Client()
<harmony.client.Client object at 0x1029e7c50>
>>>

Local Test Steps

Run Tests. 🤞

Build this package, create an environment with urllib3<2, install the package and instantiate a Request() object like demonstrated above.

PR Acceptance Checklist

  • Acceptance criteria met
  • Tests added/updated (if needed) and passing
  • Documentation updated (if needed)

This keyword was added to urllib3 in v2.0.0.
@flamingbear
Copy link
Member Author

Not sure what people think. I could check the urllib version and add it to the Request kwargs. But I'm not sure how important it is it just increases the back off time by random.uniform(0, {backoff jitter}).

Copy link
Member

@owenlittlejohns owenlittlejohns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the test instructions and could both replicate the error and show that removing backoff_jitter allowed a harmony.Client object to be instantiated. Nice.

Initially, I was a bit concerned that we don't have a unit test that caught this. But I think it's because urllib3 v2.x has been available for a few years, and is compatible with all the Python versions the CI/CD uses to execute unit tests. So we'd have to create an entirely separate environment to force the urllib3 version down to 1.26.x just for this case (seems overkill).

While I think this change makes sense, the feedback to end-users probably should still be to upgrade to urllib3 v2.x in their environments where possible. (Looks one reason for still being on v1.x is if running Python 3.9 and using boto3)

@flamingbear flamingbear merged commit 13a2878 into main Nov 24, 2025
11 checks passed
@flamingbear flamingbear deleted the mhs/remove-jitter-backoff branch November 24, 2025 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants