Visual Regression (visual-regression)

Summary

Captures multi-viewport screenshots for baseline comparison workflows (Playwright).

Classification

• Category: PERFORMANCE
• Plugin id: visual-regression
• Version: 1.0.0
• Target types: WEBSITE, WEBSHOP
• Weight class: MEDIUM
• Risk level: not set on plugin object
• Browser required: Yes (Playwright)
• Tier / group: Tier 2 — Browser (Playwright)

Schedule

• scheduleHints: { defaultInterval: '0 */6 * * *', priority: 3 }
• Prerequisites: None declared on this plugin.

Configurable inputs

Public keys are derived from inputSchema: z.object({ ... }) (underscore-prefixed keys are runtime-only and omitted here).

Field	Notes
viewports	Zod field in inputSchema — see source for defaults, min/max, and .describe().
width	Zod field in inputSchema — see source for defaults, min/max, and .describe().
height	Zod field in inputSchema — see source for defaults, min/max, and .describe().
threshold	Zod field in inputSchema — see source for defaults, min/max, and .describe().
timeout	Zod field in inputSchema — see source for defaults, min/max, and .describe().

Runtime / injected config

• _baselineDiffPercent — When set by orchestration/baseline workflow, compared against threshold to emit a regression finding.

Checks that produce findings

These checks run inside this plugin during a single test run. They are not separate tests in the database — each becomes a Finding linked to the Run for visual-regression.

Check	Typical severity	Why
Visual regression detected (_baselineDiffPercent > threshold, default 1%)	WARNING	Unexpected pixel changes may indicate broken layouts, CSS regressions, or accidental deploys on revenue-critical pages.

Without _baselineDiffPercent, screenshots are captured only — no findings are emitted.

How each check works

1. Viewport loop — Default viewports 1280×720 (desktop) and 375×812 (mobile); overridable via viewports config.
2. Capture — For each viewport: resize, networkidle goto (default 30s), 1s wait, settleVisualForScreenshot, then full-page PNG screenshot saved as screenshot-{width}x{height}.png.
3. Baseline compare (optional) — If runtime config includes _baselineDiffPercent, compares against threshold (default 1%, range 0–100). When diff exceeds threshold → WARNING finding with diff % in evidence.
4. Score and status — No baseline or diff within threshold → passed, score 100. Regression finding → warning, score 50.

Caveats and concerns

• No built-in baseline storage — Out of the box this plugin only captures artifacts; diff findings require external baseline injection via _baselineDiffPercent.
• Dynamic content — Carousels, ads, timestamps, and A/B tests cause false positives when baselines are used.
• Cookie banners — settleVisualForScreenshot dismisses common CMPs but not all; first-run vs repeat screenshots may differ.
• Full-page height — Very tall pages produce large PNGs and long capture times.
• Not a functional test — Pixel match does not validate links, forms, or accessibility.

Source

packages/test-plugins/src/plugins/visual-regression.ts