Skip to content

Conversation

teimurjan
Copy link

Description

Replaces pixelmatch third-party with a ~20-25% faster alternative BlazeDiff. It's type-safe and uses an enhanced algorithm inspired by pixelmatch. The API and results are identical.

Check Run Benchmark step.

Benchmarks

I have created a patched playwright-core package with @blazediff/core instead of pixelmatch.

Then, I created two repositories for benchmarking:

  1. Using the original Playwright: https://github.com/teimurjan/playwright-benchmark.
  2. Using the patched Playwright: https://github.com/teimurjan/playwright-blazediff-benchmark.

The benchmarks track time spent on toHaveScreenshot with 3 warmup iterations and 25 actual iterations.

Results:

  • Before (pixelmatch): 2059.12ms, 1522.12ms, 1941.32ms average times
  • After (BlazeDiff): 1987.28ms, 1476.44ms, 1893.28ms average times
  • Improvement: ~4% faster in real Playwright usage scenarios.

Closes #37267.

@teimurjan
Copy link
Author

@teimurjan please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.

@microsoft-github-policy-service agree [company="{your company}"]

Options:

  • (default - no company specified) I have sole ownership of intellectual property rights to my Submissions and I am not making Submissions in the course of work for my employer.
@microsoft-github-policy-service agree
  • (when company given) I am making Submissions in the course of work for my employer (or my employer has intellectual property rights in my Submissions by contract or applicable law). I have permission from my employer to make Submissions and enter into this Agreement on behalf of my employer. By signing below, the defined term “You” includes me and my employer.
@microsoft-github-policy-service agree company="Microsoft"

Contributor License Agreement

@microsoft-github-policy-service agree

Copy link
Contributor

github-actions bot commented Sep 2, 2025

Test results for "tests 1"

7 failed
❌ [playwright-test] › runner.spec.ts:118 › should ignore subprocess creation error because of SIGINT @macos-latest-node18-1
❌ [playwright-test] › to-have-screenshot.spec.ts:1174 › should attach expected/actual/diff when sizes are different @macos-latest-node18-1
❌ [playwright-test] › to-have-screenshot.spec.ts:1174 › should attach expected/actual/diff when sizes are different @ubuntu-latest-node18-1
❌ [playwright-test] › to-have-screenshot.spec.ts:1174 › should attach expected/actual/diff when sizes are different @ubuntu-latest-node20-1
❌ [playwright-test] › to-have-screenshot.spec.ts:1174 › should attach expected/actual/diff when sizes are different @ubuntu-latest-node22-1
❌ [playwright-test] › to-have-screenshot.spec.ts:1174 › should attach expected/actual/diff when sizes are different @ubuntu-latest-node24-1
❌ [playwright-test] › to-have-screenshot.spec.ts:1174 › should attach expected/actual/diff when sizes are different @windows-latest-node18-1

2 flaky ⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1079 › cli codegen › should not throw csp directive violation errors `@firefox-ubuntu-22.04-node18`
⚠️ [playwright-test] › ui-mode-trace.spec.ts:397 › should work behind reverse proxy `@macos-latest-node18-1`

46724 passed, 817 skipped


Merge workflow run.

@mxschmitt
Copy link
Member

I don't think we are comfortable migrating to an alternative package with 600 downloads/week. The actual one is still maintained and we haven't received a bug report about size/performance.

Closing as per above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: Replace pixelmatch with BlazeDiff for improved performance in visual screenshot testing
2 participants