Skip to content

Conversation

mxschmitt
Copy link
Member

Fixes #37274

@mxschmitt mxschmitt force-pushed the bugfix/unflake-snapshots-test branch from ea7fab7 to dca7d7b Compare September 3, 2025 13:41
@mxschmitt mxschmitt added the CQ1 label Sep 3, 2025

This comment has been minimized.

Copy link
Contributor

github-actions bot commented Sep 3, 2025

Test results for "tests 1"

2 failed
❌ [playwright-test] › runner.spec.ts:118 › should ignore subprocess creation error because of SIGINT @macos-latest-node18-1
❌ [playwright-test] › reporter-html.spec.ts:2999 › created › execSync doesnt produce a second stdout attachment @macos-latest-node18-2

4 flaky ⚠️ [chromium-library] › library/chromium/oopif.spec.ts:284 › should click `@chromium-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-ubuntu-22.04-node18`
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104 › should work with strict CSP policy `@firefox-ubuntu-22.04-node18`
⚠️ [playwright-test] › ui-mode-trace.spec.ts:637 › should indicate current test status `@ubuntu-latest-node22-1`

46735 passed, 819 skipped


Merge workflow run.

Copy link
Contributor

github-actions bot commented Sep 3, 2025

Test results for "tests 2"

6 failed
❌ [chromium-library] › library/headful.spec.ts:252 › should click bottom row w/ infobar in OOPIF @channel-chromium-macos-latest
❌ [chromium-library] › library/headful.spec.ts:252 › should click bottom row w/ infobar in OOPIF @chrome-beta-macos-latest
❌ [chromium-library] › library/headful.spec.ts:252 › should click bottom row w/ infobar in OOPIF @chrome-macos-latest
❌ [chromium-library] › library/headful.spec.ts:252 › should click bottom row w/ infobar in OOPIF @msedge-beta-macos-latest
❌ [chromium-library] › library/headful.spec.ts:252 › should click bottom row w/ infobar in OOPIF @msedge-dev-macos-latest
❌ [chromium-library] › library/headful.spec.ts:252 › should click bottom row w/ infobar in OOPIF @msedge-macos-latest

82 flaky ⚠️ [chromium-library] › library/debug-controller.spec.ts:414 › should not work with browser._launchServer(_debugController: false) `@channel-chromium-macos-latest`
⚠️ [chromium-library] › library/multiclient.spec.ts:109 › should receive viewport size changes `@chrome-beta-macos-latest`
⚠️ [chromium-library] › library/video.spec.ts:379 › screencast › should capture navigation `@chrome-beta-macos-latest`
⚠️ [chromium-page] › page/workers.spec.ts:271 › should support extra http headers `@chrome-macos-latest`
⚠️ [chromium-page] › page/workers.spec.ts:150 › should report network activity `@chromium-headed-macos-14-xlarge`
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:270 › reuse connect › should reset mouse position `@chromium-headed-ubuntu-24.04`
⚠️ [chromium-library] › library/inspector/cli-codegen-pick-locator.spec.ts:35 › should update locator highlight `@chromium-headed-ubuntu-24.04`
⚠️ [chromium-library] › library/trace-viewer.spec.ts:476 › should show snapshot URL `@chromium-headed-ubuntu-24.04`
⚠️ [chromium-page] › page/page-drag.spec.ts:290 › Drag and drop › should work with the helper method `@chromium-headed-windows-latest`
⚠️ [chromium-library] › library/trace-viewer.spec.ts:1319 › should pick locator in iframe `@chromium-macos-13-large`
⚠️ [chromium-library] › library/chromium/tracing.spec.ts:99 › should support a buffer without a path `@chromium-macos-13-xlarge`
⚠️ [chromium-page] › page/workers.spec.ts:25 › Page.workers @smoke `@chromium-tip-of-tree-macos-13`
⚠️ [chromium-page] › page/workers.spec.ts:271 › should support extra http headers `@chromium-tip-of-tree-macos-13`
⚠️ [chromium-page] › page/workers.spec.ts:25 › Page.workers @smoke `@chromium-tip-of-tree-macos-13--headed`
⚠️ [chromium-page] › page/workers.spec.ts:150 › should report network activity `@chromium-tip-of-tree-macos-13--headed`
⚠️ [chromium-library] › library/trace-viewer.spec.ts:476 › should show snapshot URL `@chromium-tip-of-tree-ubuntu-22.04--headed`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-beta-macos-latest`
⚠️ [firefox-library] › library/permissions.spec.ts:115 › permissions › should trigger permission onchange `@firefox-beta-macos-latest`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-beta-macos-latest`
⚠️ [firefox-page] › page/page-goto.spec.ts:81 › should work with Cross-Origin-Opener-Policy `@firefox-beta-macos-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-beta-ubuntu-22.04`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-beta-ubuntu-22.04`
⚠️ [firefox-library] › library/browsercontext-cookies.spec.ts:378 › should parse cookie with large Max-Age correctly `@firefox-beta-windows-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-beta-windows-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-java.spec.ts:125 › should print a valid basic program in junit `@firefox-beta-windows-latest`
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104 › should work with strict CSP policy `@firefox-beta-windows-latest`
⚠️ [firefox-library] › library/browsercontext-basic.spec.ts:361 › should emulate media in cross-process iframe `@firefox-headed-macos-14-xlarge`
⚠️ [firefox-library] › library/capabilities.spec.ts:244 › requestFullscreen `@firefox-headed-macos-14-xlarge`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-headed-macos-14-xlarge`
⚠️ [firefox-library] › library/emulation-focus.spec.ts:190 › should not fire blur events when interacting with more than one page/context `@firefox-headed-ubuntu-24.04`
⚠️ [firefox-page] › page/page-click.spec.ts:292 › should scroll and click the button with smooth scroll behavior `@firefox-headed-ubuntu-24.04`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-headed-ubuntu-24.04`
⚠️ [firefox-library] › library/browsercontext-basic.spec.ts:190 › should not report frameless pages on error `@firefox-headed-windows-latest`
⚠️ [firefox-library] › library/browsercontext-fetch.spec.ts:1099 › should support multipart/form-data and keep the order `@firefox-headed-windows-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:149 › cli codegen › should ignore programmatic events `@firefox-headed-windows-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:984 › cli codegen › should record slider `@firefox-headed-windows-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-3.spec.ts:672 › cli codegen › should consume contextmenu events, despite a custom context menu `@firefox-headed-windows-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:201 › should print context options method override in nunit if options were passed `@firefox-headed-windows-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-csharp.spec.ts:264 › should print a valid basic program in nunit `@firefox-headed-windows-latest`
⚠️ [firefox-page] › page/page-click-scroll.spec.ts:100 › should scroll into view element in iframe `@firefox-headed-windows-latest`
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104 › should work with strict CSP policy `@firefox-headed-windows-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-macos-13-large`
⚠️ [firefox-library] › library/trace-viewer.spec.ts:1319 › should pick locator in iframe `@firefox-macos-13-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-macos-13-xlarge`
⚠️ [firefox-page] › page/page-event-request.spec.ts:182 › should return response body when Cross-Origin-Opener-Policy is set `@firefox-macos-13-xlarge`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-macos-14-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-macos-14-xlarge`
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104 › should work with strict CSP policy `@firefox-macos-14-xlarge`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-ubuntu-24.04`
⚠️ [firefox-page] › page/page-goto.spec.ts:81 › should work with Cross-Origin-Opener-Policy `@firefox-ubuntu-24.04`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-windows-latest`
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104 › should work with strict CSP policy `@firefox-windows-latest`
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:150 › reuse launch › should reset serviceworker that hangs in importScripts `@msedge-beta-macos-latest`
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:303 › reuse connect › should reset tracing `@msedge-beta-macos-latest`
⚠️ [chromium-library] › library/video.spec.ts:187 › screencast › should capture static page `@msedge-beta-macos-latest`
⚠️ [chromium-page] › page/workers.spec.ts:271 › should support extra http headers `@msedge-beta-macos-latest`
⚠️ [chromium-page] › page/workers.spec.ts:150 › should report network activity `@msedge-dev-macos-latest`
⚠️ [chromium-library] › library/video.spec.ts:580 › screencast › should capture static page in persistent context @smoke `@msedge-dev-windows-latest`
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:114 › reuse connect › should reset serviceworker `@msedge-macos-latest`
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:195 › reuse connect › should not cache resources `@msedge-macos-latest`
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:250 › reuse connect › should ignore binding from beforeunload `@msedge-macos-latest`
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:355 › reuse connect › should work with routeWebSocket `@msedge-macos-latest`
⚠️ [chromium-library] › library/global-fetch.spec.ts:257 › should be able to construct with context options `@msedge-macos-latest`
⚠️ [chromium-library] › library/proxy.spec.ts:93 › should proxy local network requests › with other bypasses › localhost `@msedge-macos-latest`
⚠️ [chromium-library] › library/browsercontext-proxy.spec.ts:103 › should set cookie for top-level domain `@msedge-ubuntu-22.04`
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:265 › run-server › should support slowmo option `@tracing-chromium-tip-of-tree`
⚠️ [firefox-library] › library/page-close.spec.ts:40 › expect should not print timed out error message when page closes `@tracing-firefox`
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:104 › should work with strict CSP policy `@tracing-firefox`
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:612 › cli codegen › should check with keyboard `@webkit-headed-ubuntu-22.04`
⚠️ [webkit-library] › library/inspector/cli-codegen-2.spec.ts:250 › cli codegen › should handle dialogs `@webkit-headed-ubuntu-22.04`
⚠️ [webkit-library] › library/browsercontext-reuse.spec.ts:270 › reuse launch › should reset mouse position `@webkit-headed-ubuntu-24.04`
⚠️ [webkit-library] › library/browsercontext-reuse.spec.ts:270 › reuse connect › should reset mouse position `@webkit-headed-windows-latest`
⚠️ [webkit-library] › library/inspector/cli-codegen-pick-locator.spec.ts:35 › should update locator highlight `@webkit-headed-windows-latest`
⚠️ [webkit-library] › library/trace-viewer.spec.ts:752 › should work with meta CSP `@webkit-headed-windows-latest`
⚠️ [webkit-page] › page/page-click.spec.ts:251 › should click on checkbox input and toggle `@webkit-headed-windows-latest`
⚠️ [webkit-library] › library/trace-viewer.spec.ts:1319 › should pick locator in iframe `@webkit-macos-13-large`
⚠️ [webkit-page] › page/page-request-continue.spec.ts:205 › post data › should compute content-length from post data `@webkit-macos-14-large`
⚠️ [webkit-library] › library/trace-viewer.spec.ts:1319 › should pick locator in iframe `@webkit-macos-15-large`
⚠️ [webkit-library] › library/video.spec.ts:475 › screencast › should scale frames down to the requested size `@webkit-macos-15-large`
⚠️ [webkit-page] › page/wheel.spec.ts:70 › should dispatch wheel events after context menu was opened `@webkit-macos-15-xlarge`
⚠️ [webkit-page] › page/page-set-input-files.spec.ts:38 › should upload a folder `@webkit-ubuntu-24.04`
⚠️ [webkit-library] › library/browsercontext-reuse.spec.ts:114 › reuse launch › should reset serviceworker `@webkit-windows-latest`

253486 passed, 9960 skipped


Merge workflow run.

@mxschmitt mxschmitt merged commit 8036df3 into microsoft:main Sep 3, 2025
98 of 110 checks passed
@mxschmitt mxschmitt deleted the bugfix/unflake-snapshots-test branch September 3, 2025 15:00
Copy link
Contributor

github-actions bot commented Sep 3, 2025

Test results for "tests others"

4 fatal errors, not part of any test

2 flaky ⚠️ [electron-page] › page/locator-frame.spec.ts:272 › should work with COEP/COOP/CORP isolated iframe `@electron-macos-latest`
⚠️ [electron-page] › page/page-focus.spec.ts:81 › should traverse only form elements `@electron-macos-latest`

19477 passed, 441 skipped, 819 did not run


Merge workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[internal] library/trace-viewer.spec.ts - should handle failed snapshots due to dialog is failing
2 participants