Skip to content

Conversation

Pijukatel
Copy link
Collaborator

@Pijukatel Pijukatel commented Oct 16, 2025

Description

  • Limit cffi to version <2.0.0 to avoid increased test flakiness on Windows and Python 3.10

Issues

Testing

  • Running tests in CI and monitoring for flakiness

@github-actions github-actions bot added this to the 125th sprint - Tooling team milestone Oct 16, 2025
@github-actions github-actions bot added the t-tooling Issues with this label are in the ownership of the tooling team. label Oct 16, 2025
@Pijukatel Pijukatel changed the title Test downgrading cffi in CI chore: Fix increased flakinnes of curl_cffi based tests Oct 16, 2025
@github-actions github-actions bot added the tested Temporary label used only programatically for some analytics. label Oct 16, 2025
@Pijukatel
Copy link
Collaborator Author

Running on Windows with Python3.10 I got a lot of event loop warnings and maybe this one is specifically for curl based tests

tests/unit/http_clients/test_curl_impersonate.py: 4 warnings
D:\Workspace\Python\crawlee-python.venv\lib\site-packages\curl_cffi\aio.py:144: CurlCffiWarning:
Proactor event loop does not implement add_reader family of methods required.
Registering an additional selector thread for add_reader support.
To avoid this warning use:
asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())

Given that, maybe it would be best to use WindowsSelectorEventLoopPolicy all the time in our tests

@Pijukatel
Copy link
Collaborator Author

Randomizing test order is enough to make UTs stable. This means that some tests are not properly isolated. This could cause some race conditions for shared resources. Maybe the session-scoped test server together with xdist is the problem, too hard to debug and most likely not a product issue, but test setup issue.

@Pijukatel Pijukatel closed this Oct 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix increased flakinnes of curl_cffi based tests on Windows and Python 3.10

1 participant