-
Notifications
You must be signed in to change notification settings - Fork 52
Fix CI and VRT flakiness and tech debt #1119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
🟢 No design token changes found |
🟢 No visual differences foundOur visual comparison tests did not find any differences in the UI. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR modernizes the visual regression testing approach for improved stability and reliability by switching from the deprecated toMatchSnapshot
method to the newer toHaveScreenshot
method across all Playwright visual test files.
- Switch from
toMatchSnapshot
totoHaveScreenshot
for all visual regression tests - Update test ordering in PricingOptions tests by reordering "Leading Component" and "One Option" test groups
- Maintain consistent test structure and timeout patterns across all affected files
Reviewed Changes
Copilot reviewed 37 out of 126 changed files in this pull request and generated no comments.
Show a summary per file
File | Description |
---|---|
packages/react/src/Prose/Prose.visual.spec.ts | Updated screenshot assertions from toMatchSnapshot to toHaveScreenshot |
packages/react/src/PricingOptions/PricingOptions.visual.spec.ts | Updated screenshot assertions and reordered test groups |
packages/react/src/Pillar/Pillar.visual.spec.ts | Updated screenshot assertions from toMatchSnapshot to toHaveScreenshot |
packages/react/src/Pagination/Pagination.visual.spec.ts | Updated screenshot assertions from toMatchSnapshot to toHaveScreenshot |
packages/react/src/MonaSans/MonaSans.visual.spec.ts | Updated screenshot assertions from toMatchSnapshot to toHaveScreenshot |
packages/react/src/MinimalFooter/MinimalFooter.visual.spec.ts | Updated screenshot assertions from toMatchSnapshot to toHaveScreenshot |
packages/react/src/LogoSuite/LogoSuite.visual.spec.ts | Updated screenshot assertions from toMatchSnapshot to toHaveScreenshot |
packages/react/src/Link/Link.visual.spec.ts | Updated screenshot assertions from toMatchSnapshot to toHaveScreenshot |
packages/react/src/Label/Label.visual.spec.ts | Updated screenshot assertions from toMatchSnapshot to toHaveScreenshot |
packages/react/src/InlineLink/InlineLink.visual.spec.ts | Updated screenshot assertions from toMatchSnapshot to toHaveScreenshot |
Summary
Addresses the recent flakiness and instability in CI and VRT
List of notable changes:
toMatchSnapshot
totoHaveScreenshot
as the latter is now the preferred method for screenshot comparison using Playwright.maxDiffPixels
tomaxDiffPixelRatio
to prefer a percentage-based tolerance that scales better with image size. Big images previously had fase positives becausemaxDiffPixels
value was suitable only for smaller images. Now uses a0.02%
diff threshold. This is very aggressive, but seems to be stable enough over multiple VRT runs.Contributor checklist:
update snapshots
label to the PR)Reviewer checklist: