Skip to content

test: speculative fix for flaky role selectors test #36767

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 23, 2025

Conversation

mxschmitt
Copy link
Member

No description provided.

@mxschmitt mxschmitt added the CQ1 label Jul 23, 2025
Copy link
Contributor

Test results for "tests others"

1 fatal errors, not part of any test
20173 passed, 473 skipped
✔️✔️✔️

Merge workflow run.

@mxschmitt mxschmitt marked this pull request as ready for review July 23, 2025 15:51
Copy link
Contributor

Test results for "tests 1"

5 flaky ⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-ubuntu-22.04-node18
⚠️ [firefox-page] › page/page-event-request.spec.ts:182:3 › should return response body when Cross-Origin-Opener-Policy is set @firefox-ubuntu-22.04-node18
⚠️ [playwright-test] › ui-mode-test-watch.spec.ts:145:5 › should watch all @ubuntu-latest-node18-1
⚠️ [webkit-page] › page/elementhandle-bounding-box.spec.ts:22:3 › should work @webkit-ubuntu-22.04-node18
⚠️ [playwright-test] › ui-mode-test-watch.spec.ts:145:5 › should watch all @windows-latest-node18-1

46782 passed, 926 skipped
✔️✔️✔️

Merge workflow run.

Copy link
Contributor

Test results for "tests 2"

6 fatal errors, not part of any test
4 failed
❌ [chromium-library] › library/emulation-focus.spec.ts:202:12 › should trigger hover state concurrently @channel-chromium-macos-latest
❌ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:93:7 › cli codegen › should not lead to an error if html gets clicked @channel-chromium-macos-latest
❌ [chromium-library] › library/downloads-path.spec.ts:63:5 › downloads path › should report downloads in downloadsPath folder @msedge-beta-macos-latest
❌ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:94:7 › cli codegen › should click twice @webkit-headed-ubuntu-22.04

124 flaky ⚠️ [chromium-library] › library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @channel-chromium-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:166:1 › should highlight all @channel-chromium-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:28:3 › context.cookies() should work @smoke @channel-chromium-macos-latest
⚠️ [chromium-library] › library/download.spec.ts:389:5 › download event › should delete downloads on browser gone @channel-chromium-macos-latest
⚠️ [chromium-library] › library/downloads-path.spec.ts:32:5 › downloads path › should keep downloadsPath folder @channel-chromium-macos-latest
⚠️ [chromium-library] › library/downloads-path.spec.ts:105:5 › downloads path › should delete downloads when persistent context closes @channel-chromium-macos-latest
⚠️ [chromium-library] › library/fetch-proxy.spec.ts:21:3 › context request should pick up proxy credentials @channel-chromium-macos-latest
⚠️ [chromium-page] › page/workers.spec.ts:150:3 › should report network activity @channel-chromium-macos-latest
⚠️ [chromium-library] › library/trace-viewer.spec.ts:1351:1 › should highlight locator in iframe while typing @channel-chromium-windows-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @chrome-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:150:5 › reuse launch › should reset serviceworker that hangs in importScripts @chrome-beta-macos-latest
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:600:5 › launchServer › should be able to connect when the wsEndpoint is passed as an option @chrome-beta-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:138:1 › should navigate all @chrome-beta-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:28:3 › context.cookies() should work @smoke @chrome-beta-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:93:7 › cli codegen › should not lead to an error if html gets clicked @chrome-beta-macos-latest
⚠️ [chromium-library] › library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @chrome-macos-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @chrome-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:94:5 › reuse launch › should re-add binding after reset @chrome-macos-latest
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:647:5 › launchServer › should record trace with sources @chrome-macos-latest
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:250:5 › run-server › should send default User-Agent and X-Playwright-Browser headers with connect request @chrome-macos-latest
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:634:5 › should throw when connecting twice to an already running persistent context (--remote-debugging-port) @chrome-macos-latest
⚠️ [chromium-library] › library/client-certificates.spec.ts:705:3 › browser › support http2 if the browser only supports http1.1 @chrome-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:138:1 › should navigate all @chrome-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:28:3 › context.cookies() should work @smoke @chrome-macos-latest
⚠️ [chromium-library] › library/emulation-focus.spec.ts:202:12 › should trigger hover state concurrently @chrome-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:93:7 › cli codegen › should not lead to an error if html gets clicked @chrome-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-aria.spec.ts:76:7 › should update aria snapshot highlight @chrome-macos-latest
⚠️ [chromium-library] › library/defaultbrowsercontext-2.spec.ts:48:3 › should support forcedColors option @chromium-headed-macos-14-xlarge
⚠️ [chromium-page] › page/workers.spec.ts:271:3 › should support extra http headers @chromium-headed-macos-14-xlarge
⚠️ [chromium-library] › library/inspector/cli-codegen-1.spec.ts:434:7 › cli codegen › should press @chromium-headed-ubuntu-24.04
⚠️ [chromium-page] › page/page-drag.spec.ts:56:5 › Drag and drop › should not send dragover on the first mousemove @chromium-headed-windows-latest
⚠️ [chromium-library] › library/chromium/tracing.spec.ts:99:3 › should support a buffer without a path @chromium-macos-13-xlarge
⚠️ [chromium-page] › page/workers.spec.ts:25:3 › Page.workers @smoke @chromium-tip-of-tree-macos-13
⚠️ [chromium-page] › page/workers.spec.ts:150:3 › should report network activity @chromium-tip-of-tree-macos-13
⚠️ [chromium-page] › page/workers.spec.ts:271:3 › should support extra http headers @chromium-tip-of-tree-macos-13
⚠️ [chromium-page] › page/workers.spec.ts:150:3 › should report network activity @chromium-tip-of-tree-macos-13--headed
⚠️ [chromium-page] › page/workers.spec.ts:271:3 › should support extra http headers @chromium-tip-of-tree-macos-13--headed
⚠️ [chromium-page] › page/page-drag.spec.ts:73:5 › Drag and drop › should work inside iframe @chromium-tip-of-tree-windows-latest--headed
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-beta-macos-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-beta-ubuntu-22.04
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-beta-windows-latest
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-beta-windows-latest
⚠️ [firefox-library] › library/capabilities.spec.ts:244:3 › requestFullscreen @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › library/trace-viewer.spec.ts:1636:1 › canvas disabled title @firefox-headed-macos-14-xlarge
⚠️ [firefox-page] › page/page-goto.spec.ts:81:3 › should work with Cross-Origin-Opener-Policy @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › library/browsercontext-basic.spec.ts:36:3 › should be able to click across browser contexts @firefox-headed-ubuntu-24.04
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-headed-ubuntu-24.04
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-headed-ubuntu-24.04
⚠️ [firefox-library] › library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @firefox-headed-windows-latest
⚠️ [firefox-library] › library/browsercontext-clearcookies.spec.ts:72:3 › should remove cookies by name regex @firefox-headed-windows-latest
⚠️ [firefox-library] › library/browsercontext-har.spec.ts:399:3 › should update har.zip for context @firefox-headed-windows-latest
⚠️ [firefox-library] › library/browsertype-launch.spec.ts:133:3 › should allow await using @firefox-headed-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:653:7 › cli codegen › should uncheck @firefox-headed-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:984:7 › cli codegen › should record slider @firefox-headed-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-pick-locator.spec.ts:35:7 › should update locator highlight @firefox-headed-windows-latest
⚠️ [firefox-library] › library/trace-viewer.spec.ts:874:1 › should highlight target elements @firefox-headed-windows-latest
⚠️ [firefox-library] › library/video.spec.ts:379:5 › screencast › should capture navigation @firefox-headed-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-macos-13-large
⚠️ [firefox-library] › library/video.spec.ts:379:5 › screencast › should capture navigation @firefox-macos-13-large
⚠️ [firefox-page] › page/page-event-request.spec.ts:182:3 › should return response body when Cross-Origin-Opener-Policy is set @firefox-macos-13-large
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-macos-13-xlarge
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-macos-13-xlarge
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-macos-14-large
⚠️ [firefox-library] › library/inspector/cli-codegen-2.spec.ts:454:7 › cli codegen › should save assets via SIGINT @firefox-macos-14-large
⚠️ [firefox-page] › page/page-aria-snapshot-ai.spec.ts:358:3 › return empty snapshot when iframe is not loaded @firefox-macos-14-large
⚠️ [firefox-page] › page/page-event-request.spec.ts:182:3 › should return response body when Cross-Origin-Opener-Policy is set @firefox-macos-14-large
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-macos-14-xlarge
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-macos-14-xlarge
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-ubuntu-24.04
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-ubuntu-24.04
⚠️ [firefox-library] › library/debug-controller.spec.ts:79:1 › should pick element @firefox-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @firefox-windows-latest
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104:3 › should work with strict CSP policy @firefox-windows-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @msedge-beta-macos-latest
⚠️ [chromium-library] › library/downloads-path.spec.ts:77:5 › downloads path › should report downloads in downloadsPath folder with a relative path @msedge-beta-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:454:7 › cli codegen › should save assets via SIGINT @msedge-beta-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-beta-macos-latest
⚠️ [chromium-library] › library/proxy.spec.ts:291:3 › should use SOCKS proxy for websocket requests @msedge-beta-macos-latest
⚠️ [chromium-library] › library/proxy.spec.ts:326:3 › should use http proxy for websocket requests @msedge-beta-macos-latest
⚠️ [chromium-library] › library/tracing.spec.ts:369:5 › should survive browser.close with auto-created traces dir @msedge-beta-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-beta-ubuntu-22.04
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:150:5 › reuse connect › should reset serviceworker that hangs in importScripts @msedge-beta-windows-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:79:1 › should pick element @msedge-beta-windows-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-beta-windows-latest
⚠️ [chromium-library] › library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-fetch.spec.ts:1262:3 › should work with connectOverCDP @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-proxy.spec.ts:27:3 › should work when passing the proxy only on the context level @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:150:5 › reuse launch › should reset serviceworker that hangs in importScripts @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:250:5 › reuse launch › should ignore binding from beforeunload @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:142:5 › launchServer › should be able to reconnect to a browser @msedge-dev-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:149:1 › should reset for reuse @msedge-dev-macos-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:249:1 › should reset routes before reuse @msedge-dev-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-dev-macos-latest
⚠️ [chromium-library] › library/video.spec.ts:379:5 › screencast › should capture navigation @msedge-dev-macos-latest
⚠️ [chromium-page] › page/workers.spec.ts:150:3 › should report network activity @msedge-dev-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-dev-ubuntu-22.04
⚠️ [chromium-library] › library/browsercontext-proxy.spec.ts:103:3 › should set cookie for top-level domain @msedge-dev-windows-latest
⚠️ [chromium-library] › library/debug-controller.spec.ts:79:1 › should pick element @msedge-dev-windows-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-dev-windows-latest
⚠️ [chromium-library] › library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:150:5 › reuse launch › should reset serviceworker that hangs in importScripts @msedge-macos-latest
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:188:5 › launchServer › should be able to visit ipv6 through localhost @msedge-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:454:7 › cli codegen › should save assets via SIGINT @msedge-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-aria.spec.ts:76:7 › should update aria snapshot highlight @msedge-macos-latest
⚠️ [chromium-library] › library/logger.spec.ts:19:3 › should log @smoke @msedge-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-macos-latest
⚠️ [chromium-page] › page/workers.spec.ts:271:3 › should support extra http headers @msedge-macos-latest
⚠️ [chromium-library] › library/permissions.spec.ts:244:5 › should be able to use the local-fonts API @msedge-ubuntu-22.04
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079:7 › cli codegen › should not throw csp directive violation errors @tracing-firefox
⚠️ [firefox-page] › page/page-event-request.spec.ts:182:3 › should return response body when Cross-Origin-Opener-Policy is set @tracing-firefox
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:120:7 › cli codegen › should type after clicking twice @webkit-headed-ubuntu-22.04
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:1023:7 › cli codegen › should click button with nested div @webkit-headed-ubuntu-22.04
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:120:7 › cli codegen › should type after clicking twice @webkit-headed-ubuntu-24.04
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:1023:7 › cli codegen › should click button with nested div @webkit-headed-ubuntu-24.04
⚠️ [webkit-library] › library/inspector/cli-codegen-3.spec.ts:640:7 › cli codegen › should consume pointer events @webkit-headed-ubuntu-24.04
⚠️ [webkit-library] › library/inspector/cli-codegen-pick-locator.spec.ts:35:7 › should update locator highlight @webkit-headed-windows-latest
⚠️ [webkit-library] › library/selector-generator.spec.ts:550:5 › selector generator › should generate exact title when necessary @webkit-headed-windows-latest
⚠️ [webkit-library] › library/tracing.spec.ts:431:14 › should produce screencast frames fit @webkit-macos-14-xlarge
⚠️ [webkit-library] › library/video.spec.ts:379:5 › screencast › should capture navigation @webkit-macos-15-large
⚠️ [webkit-library] › library/inspector/cli-codegen-aria.spec.ts:76:7 › should update aria snapshot highlight @webkit-macos-15-xlarge
⚠️ [webkit-library] › library/browsercontext-reuse.spec.ts:114:5 › reuse launch › should reset serviceworker @webkit-windows-latest

244040 passed, 9680 skipped
✔️✔️✔️

Merge workflow run.

@mxschmitt mxschmitt merged commit 5e73040 into microsoft:main Jul 23, 2025
100 of 110 checks passed
@mxschmitt mxschmitt deleted the tests/speculative-fix branch July 23, 2025 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants