fix/slow_pythonapi_tests #1289

SkylordA · 2025-10-22T09:24:15Z

fix/slow_pythonapi_tests

Python api unittests are much slower after the 2.4.7 release, specifically the django 5 PR. They take ~1hr to run rather than ~20min. There is no one specific test that is slow, rather a large number of tests take longer to run each example via hypothesis. This PR aims to fix the slow runtime.

…hesis to 30

…lt iterations to speed up unit tests

SkylordA · 2025-10-27T11:49:56Z

Just adding the cProfile outputs for 2.4.6 vs main for running just 1 api test. What is important is the percall runtime for {built-in method _hashlib.pbkdf2_hmac}, being 0.041 vs 0.155.

You can also view the default iterations the PBKDF2PasswordHasher here for both versions:

2.4.6

Mon Oct 27 08:34:16 2025    profile_old

         30122734 function calls (28755116 primitive calls) in 23.726 seconds

   Ordered by: cumulative time
   List reduced from 20272 to 50 due to restriction <50>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   3288/1    0.038    0.000   23.777   23.777 {built-in method builtins.exec}
        1    0.000    0.000   23.777   23.777 <string>:1(<module>)
        1    0.000    0.000   23.777   23.777 <frozen runpy>:201(run_module)
        1    0.000    0.000   23.613   23.613 <frozen runpy>:65(_run_code)
        1    0.000    0.000   23.613   23.613 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pytest/__main__.py:1(<module>)
        1    0.000    0.000   23.613   23.613 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/config/__init__.py:194(console_main)
        1    0.000    0.000   23.613   23.613 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/config/__init__.py:139(main)
  2404/71    0.001    0.000   23.588    0.332 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pluggy/_manager.py:111(_hookexec)
  2404/71    0.003    0.000   23.588    0.332 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pluggy/_callers.py:53(_multicall)
    385/2    0.001    0.000   23.586   11.793 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pluggy/_hooks.py:498(__call__)
        1    0.000    0.000   21.268   21.268 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/main.py:329(pytest_cmdline_main)
        1    0.000    0.000   21.268   21.268 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/main.py:270(wrap_session)
        1    0.000    0.000   21.242   21.242 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/main.py:333(_main)
     13/4    0.000    0.000   21.223    5.306 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:319(from_call)
        1    0.000    0.000   20.436   20.436 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/main.py:350(pytest_runtestloop)
        1    0.000    0.000   20.420   20.420 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:110(pytest_runtest_protocol)
        1    0.000    0.000   20.420   20.420 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:118(runtestprotocol)
        3    0.000    0.000   20.420    6.807 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:226(call_and_report)
        3    0.000    0.000   20.419    6.806 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:242(<lambda>)
        1    0.000    0.000   18.165   18.165 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:163(pytest_runtest_call)
        1    0.000    0.000   18.165   18.165 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pytest_django/plugin.py:553(non_debugging_runtest)
        1    0.000    0.000   18.165   18.165 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django_webtest/__init__.py:334(__call__)
        1    0.000    0.000   18.156   18.156 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/extra/django/_impl.py:43(__call__)
        1    0.000    0.000   18.156   18.156 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/unittest/case.py:689(__call__)
        1    0.000    0.000   18.156   18.156 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/unittest/case.py:599(run)
        1    0.000    0.000   18.156   18.156 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/unittest/case.py:588(_callTestMethod)
        1    0.000    0.000   18.156   18.156 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:1507(wrapped_test)
        1    0.000    0.000   18.149   18.149 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:1133(run_engine)
        1    0.000    0.000   18.147   18.147 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:776(run)
        1    0.000    0.000   18.147   18.147 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:1239(_run)
      100    0.002    0.000   18.047    0.180 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:417(test_function)
      100    0.000    0.000   18.028    0.180 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:307(__stoppable_test_function)
      100    0.001    0.000   18.026    0.180 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:995(_execute_once_for_engine)
      100    0.002    0.000   18.025    0.180 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:782(execute_once)
      100    0.000    0.000   17.988    0.180 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:725(execute)
      100    0.001    0.000   17.943    0.179 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:849(run)
      100    0.002    0.000   17.871    0.179 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:816(test)
        1    0.000    0.000   17.844   17.844 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:912(generate_new_examples)
      200    0.001    0.000    8.429    0.042 /home/anish/Documents/github/OasisPlatform/src/server/oasisapi/auth/tests/fakes.py:8(fake_user)
      500    0.005    0.000    8.191    0.016 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django_webtest/__init__.py:77(do_request)
      200    0.000    0.000    8.191    0.041 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django/contrib/auth/base_user.py:98(set_password)
      200    0.001    0.000    8.191    0.041 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django/contrib/auth/hashers.py:65(make_password)
      200    0.001    0.000    8.177    0.041 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django/contrib/auth/hashers.py:271(encode)
      200    0.001    0.000    8.175    0.041 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django/utils/crypto.py:80(pbkdf2)
      200    8.174    0.041    8.174    0.041 {built-in method _hashlib.pbkdf2_hmac}
      500    0.007    0.000    8.160    0.016 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/webtest/app.py:591(do_request)
      500    0.002    0.000    8.106    0.016 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/webob/request.py:1294(send)
      500    0.001    0.000    8.099    0.016 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/webob/request.py:1256(call_application)
      500    0.003    0.000    8.097    0.016 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/webtest/lint.py:163(lint_app)
      500    0.001    0.000    8.086    0.016 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django/contrib/staticfiles/handlers.py:74(__call__)

main (after the 2.4.7 Django 5 update):

Mon Oct 27 08:28:03 2025    profile

         32957578 function calls (31831635 primitive calls) in 46.236 seconds

   Ordered by: cumulative time
   List reduced from 18634 to 50 due to restriction <50>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   3156/1    0.026    0.000   46.280   46.280 {built-in method builtins.exec}
        1    0.000    0.000   46.280   46.280 <string>:1(<module>)
        1    0.000    0.000   46.280   46.280 <frozen runpy>:201(run_module)
        1    0.000    0.000   46.199   46.199 <frozen runpy>:65(_run_code)
        1    0.000    0.000   46.199   46.199 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pytest/__main__.py:1(<module>)
        1    0.000    0.000   46.199   46.199 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/config/__init__.py:194(console_main)
        1    0.000    0.000   46.199   46.199 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/config/__init__.py:139(main)
   565/69    0.000    0.000   46.186    0.669 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pluggy/_manager.py:111(_hookexec)
   565/69    0.001    0.000   46.186    0.669 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pluggy/_callers.py:76(_multicall)
    382/2    0.001    0.000   46.184   23.092 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pluggy/_hooks.py:497(__call__)
        1    0.000    0.000   44.553   44.553 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/main.py:335(pytest_cmdline_main)
        1    0.000    0.000   44.553   44.553 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/main.py:276(wrap_session)
        1    0.000    0.000   44.245   44.245 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/main.py:339(_main)
     13/4    0.000    0.000   44.243   11.061 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:323(from_call)
        1    0.000    0.000   44.169   44.169 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/main.py:356(pytest_runtestloop)
        1    0.000    0.000   44.168   44.168 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:114(pytest_runtest_protocol)
        1    0.000    0.000   44.168   44.168 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:122(runtestprotocol)
        3    0.000    0.000   44.168   14.723 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:230(call_and_report)
        3    0.000    0.000   44.167   14.722 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:246(<lambda>)
        1    0.000    0.000   42.044   42.044 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/_pytest/runner.py:167(pytest_runtest_call)
        1    0.000    0.000   42.044   42.044 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/pytest_django/plugin.py:572(non_debugging_runtest)
        1    0.000    0.000   42.044   42.044 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django_webtest/__init__.py:334(__call__)
        1    0.000    0.000   42.037   42.037 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/extra/django/_impl.py:36(__call__)
        1    0.000    0.000   42.037   42.037 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/unittest/case.py:689(__call__)
        1    0.000    0.000   42.037   42.037 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/unittest/case.py:599(run)
        1    0.000    0.000   42.037   42.037 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/unittest/case.py:588(_callTestMethod)
        1    0.000    0.000   42.037   42.037 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:1899(wrapped_test)
        1    0.000    0.000   42.029   42.029 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:1374(run_engine)
        1    0.000    0.000   42.027   42.027 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:922(run)
        1    0.000    0.000   42.027   42.027 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:1486(_run)
        1    0.000    0.000   42.027   42.027 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:1090(generate_new_examples)
      100    0.002    0.000   41.988    0.420 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:510(test_function)
      100    0.000    0.000   41.977    0.420 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:402(__stoppable_test_function)
      100    0.001    0.000   41.975    0.420 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:1197(_execute_once_for_engine)
      100    0.002    0.000   41.974    0.420 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:964(execute_once)
      100    0.000    0.000   41.934    0.419 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:837(execute)
      100    0.001    0.000   41.892    0.419 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:1044(run)
      100    0.002    0.000   41.605    0.416 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/core.py:1002(test)
      200    0.002    0.000   31.215    0.156 /home/anish/Documents/github/OasisPlatform/src/server/oasisapi/auth/tests/fakes.py:8(fake_user)
      200    0.000    0.000   30.957    0.155 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django/contrib/auth/base_user.py:96(set_password)
      200    0.001    0.000   30.957    0.155 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django/contrib/auth/hashers.py:94(make_password)
      200    0.002    0.000   30.944    0.155 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django/contrib/auth/hashers.py:324(encode)
      200    0.001    0.000   30.941    0.155 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django/utils/crypto.py:70(pbkdf2)
      200   30.939    0.155   30.939    0.155 {built-in method _hashlib.pbkdf2_hmac}
      315    0.001    0.000   21.328    0.068 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:433(cached_test_function)
       49    0.002    0.000   18.342    0.374 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/hypothesis/internal/conjecture/engine.py:1282(generate_mutations_from)
      500    0.005    0.000    9.269    0.019 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/django_webtest/__init__.py:77(do_request)
      500    0.006    0.000    9.226    0.018 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/webtest/app.py:591(do_request)
      500    0.002    0.000    9.170    0.018 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/webob/request.py:1294(send)
      500    0.001    0.000    9.164    0.018 /home/anish/anaconda3/envs/oasis_platform/lib/python3.12/site-packages/webob/request.py:1256(call_application)

adds hack to slow python unittests by limiting max_examples for hypot…

636ff47

…hesis to 30

SkylordA self-assigned this Oct 22, 2025

SkylordA added the bug label Oct 22, 2025

SkylordA linked an issue Oct 22, 2025 that may be closed by this pull request

python unit testing takes 1h #1269

Closed

awsbuild added this to Oasis Dev Team Tasks Oct 22, 2025

SkylordA added 5 commits October 22, 2025 10:27

adds hypothesis to requirements

d220d41

pip compile requirements

57f2103

removes conftest.py, creates a custom passwordhasher with fewer defau…

c373f33

…lt iterations to speed up unit tests

makes FastPBKDF2PasswordHasher default only for unittests

e507c2a

undo changes to worker-controller requirements

71f0e94

SkylordA requested a review from sambles October 29, 2025 11:46

Merge branch 'main' into fix/slow_pythonapi_tests

5bd890c

sambles approved these changes Nov 6, 2025

View reviewed changes

sambles merged commit 552045f into main Nov 6, 2025
32 checks passed

sambles deleted the fix/slow_pythonapi_tests branch November 6, 2025 08:58

github-project-automation bot moved this to Done in Oasis Dev Team Tasks Nov 6, 2025

sambles mentioned this pull request Nov 7, 2025

Release 2.4.9 #1292

Merged

awsbuild added this to the 2.4.9 milestone Nov 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix/slow_pythonapi_tests #1289

fix/slow_pythonapi_tests #1289

Uh oh!

SkylordA commented Oct 22, 2025

Uh oh!

SkylordA commented Oct 27, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fix/slow_pythonapi_tests #1289

fix/slow_pythonapi_tests #1289

Uh oh!

Conversation

SkylordA commented Oct 22, 2025