Storybook Visual-Regression Harness

This document describes the visual-regression harness wired into CI. The harness ships the static Storybook bundle to Chromatic on every pull request and surfaces pixel-level diffs against an approved baseline so a regression reviewer can accept or reject the change before merge.

It supplements — and does not replace — the existing Vitest unit suite, Playwright end-to-end suite, and npm run test:a11y accessibility gate. Visual regression catches the class of regression none of those can: unintended CSS spacing changes, accidental theme-token drift, dark-mode breakage, and component-level layout shifts that compile-and-render cleanly but visibly differ from the approved design.

What's in the harness

Storybook 8.4.x scaffold under .storybook/. The bundle is built by npm run build-storybook and lands at storybook-static/ in the workspace root.
Story authoring under src/components/**/*.stories.tsx (starting with the host-pool primitive stories). The Storybook stories glob picks up any future story file automatically — no per-story registration is needed.
CI job storybook-chromatic in vetrix-ci.yml at the repo root. It runs npm ci --legacy-peer-deps, builds the Storybook bundle, and uploads to Chromatic via the chromatic CLI.
This document describing the baseline-approval workflow, reviewer responsibilities, and operator setup.

Why Chromatic

Chromatic fits this harness for the following reasons:

First-class Storybook integration. Chromatic is published by the Storybook team. The chromatic CLI knows how to read the Storybook 8.x manifest, snapshot every story with deterministic viewports, and skip stories that ship with the chromatic: { disableSnapshot: true } parameter.
Server-side snapshot rendering. Snapshots are captured on Chromatic's infrastructure, not on the Vetrix runner, so the CI job stays small (build + upload) and snapshot consistency is decoupled from the runner's font / GPU / browser stack.
PR check status. Chromatic posts a status check directly to the PR with a deep link to the diff review UI; reviewers do not have to download artifacts to triage a regression.
Free tier. Chromatic's free tier covers 5,000 snapshots per month — comfortably enough for the in-flight story library while the team builds coverage.

If the team later outgrows Chromatic (snapshot budget, single-vendor risk), the harness is portable: the CI job is one block in vetrix-ci.yml and the static Storybook bundle is artifact-published, so swapping the upload target is a one-file change.

Operator setup (one-time)

These steps cannot be performed by the development pipeline — they require a human with operator-level access to Chromatic and the Vetrix runner secret store.

1. Provision the Chromatic project

Sign in (or create an account) at https://www.chromatic.com using the operator email associated with the vetrix-frontend repository.
Add a new project. Choose "GitHub-like git provider" (Chromatic does not have first-class Vetrix integration; the project-token flow does not depend on a webhook).
Name the project vetrix-frontend. Pick the free tier; upgrade later if the snapshot budget is exceeded.
On the project's "Manage" → "Configure" page copy the project token (a UUID-shaped string starting with chpt_…).

2. Inject the token into the Vetrix runner

The Vetrix runner reads job-level ${VAR_NAME} interpolations from its runtime environment. Operators inject CHROMATIC_PROJECT_TOKEN by setting it as a runner-side environment variable on the Vetrix runner that picks up vetrix-frontend jobs:

Docker-runner deployment: add an --env flag to the runner's container start command (or set it in the runner's compose file's environment: block):

docker run -d --name vetrix-runner-frontend \
  -e CHROMATIC_PROJECT_TOKEN="chpt_xxxxxxxxxxxxxxxxxxxxxxxx" \
  -v /var/run/docker.sock:/var/run/docker.sock \
  vetrix-runner:latest

Kubernetes-runner deployment: store the token in a Secret, then reference it as an env entry on the runner Deployment:

apiVersion: v1
kind: Secret
metadata:
  name: vetrix-runner-secrets
stringData:
  CHROMATIC_PROJECT_TOKEN: "chpt_xxxxxxxxxxxxxxxxxxxxxxxx"
---
# in the runner Deployment
env:
  - name: CHROMATIC_PROJECT_TOKEN
    valueFrom:
      secretKeyRef:
        name: vetrix-runner-secrets
        key: CHROMATIC_PROJECT_TOKEN

Restart the runner after either change so the new env reaches the runner process. The next pipeline that schedules the storybook-chromatic job will pick up the token automatically; the job is marked secret: true in vetrix-ci.yml so the value is redacted from log output.

3. Confirm the upload works

Open any PR that targets develop or an rc/* branch. The storybook-chromatic job runs in the test stage. The pipeline log should show:

npm ci --legacy-peer-deps succeeds.
npm run build-storybook produces storybook-static/.
The chromatic CLI prints a build URL of the form https://www.chromatic.com/build?appId=…&number=….
The job exits 0.

The first build of any project is also the first baseline: Chromatic auto-accepts every snapshot in the initial build so subsequent PRs have a known-good reference to diff against. There is no review required for the first build.

4. Until step 2 is complete

The CI job is intentionally tolerant of the pre-wire window. If CHROMATIC_PROJECT_TOKEN is not yet set on the runner, the ${CHROMATIC_PROJECT_TOKEN} interpolation resolves to the literal string ${CHROMATIC_PROJECT_TOKEN} (per the Vetrix parser: "Undefined variables are left as-is"). The job detects that literal, prints a one-paragraph banner pointing at this document, and exits 0 so PRs are not blocked while operator setup is pending.

The job is also marked non-blocking (allow_failure: true) for the initial-coverage window — see Non-blocking gate below.

Baseline-approval workflow (reviewer-facing)

Once the operator has completed setup and the first baseline has been recorded, every PR that changes a Storybook-rendered component will either pass cleanly or produce a diff that needs human approval.

When Chromatic reports "0 changes"

The job logs storybook-chromatic: upload clean (no blocking diffs) and the PR's Chromatic status check is green. No reviewer action is required.

When Chromatic reports diffs

The PR's Chromatic status check links to a build page on the Chromatic UI. The build page lists every story whose rendered pixels differ from the current baseline. For each diff:

Open the diff in the Chromatic UI. The side-by-side viewer shows the baseline, the new render, and a per-pixel overlay.
Decide intent:
- Accept — the change is intentional (a redesign, a token swap, a layout intent change). Approve the diff in the Chromatic UI; the new render becomes the baseline for the story on this branch. After merge, the baseline promotes to develop automatically.
- Reject — the change is unintentional (the diff is a regression). Reject in the Chromatic UI and bounce the PR back to the author with a link to the Chromatic build. The author either fixes the regression (re-run the job, re-review) or, if the change really is intentional and the bounce was misjudged, re-pushes and asks for re-review.
Approve story-by-story, not build-by-build. Chromatic supports bulk-accept but the team's policy is per-story so accidental acceptance of an adjacent regression cannot piggy-back on an intentional change.

The Chromatic UI emails the PR author on every decision; the status-check colour on the PR follows the per-story state.

Who approves

Author of the story — by default, the dev who shipped the story approves diffs against it.
Design system steward — for shared primitives (anything under src/components/ui/ or theme-token consumers), the design-system steward of record (currently the team's frontend lead) is the required approver. Mark these stories with the chromatic: { reviewer: 'design-system' } parameter in the story file so the Chromatic UI surfaces the requirement.
Anyone during the bootstrap phase — until the team builds out the review-routing rules, any active maintainer may approve any diff. Document the approver in the PR comment.

What gets a story

Every component that ships visible UI deserves at least one story. Until coverage is comprehensive, the absence of a story is not a blocker on a PR — but adding stories alongside new components is encouraged.

Non-blocking gate

The storybook-chromatic job declares the native per-job key allow_failure: true in vetrix-ci.yml. This means:

A chromatic-CLI exit code of zero passes the job (normal case).
A non-zero chromatic exit code (e.g., the project token is rejected, the upload fails for a network reason, or Chromatic reports an infrastructure failure) propagates: the job logs a clear banner, exits non-zero, and reports failed (soft) at the pipeline summary and commit-status surface (the soft_failed projection) — without failing the pipeline.
The unset-token no-op is unchanged: when CHROMATIC_PROJECT_TOKEN is not wired on the runner, the job detects the surviving ${CHROMATIC_PROJECT_TOKEN} literal and exits 0 — a genuine config-absent skip (no-op pass), not a soft failure. See Until step 2 is complete.
The --exit-zero-on-changes flag passed to the chromatic CLI additionally tells Chromatic to exit 0 even when visual diffs are detected; reviewers see the diffs in the Chromatic UI and review them out-of-band, but the CI job itself does not turn red on a diff.

The intent is to ship the harness without disrupting the merge workflow while coverage stabilises. Once visual coverage is comprehensive and the team's tolerance for diff-driven CI failures is established, delete the allow_failure: true line from the storybook-chromatic job in vetrix-ci.yml (single-line edit) to promote the gate to enforcing.

allow_failure: <bool> is a first-class key in Vetrix's v1 CI schema; a failing job with allow_failure: true soft-fails. There is no SOFT_FAIL variable anywhere in vetrix-ci.yml; a build guard under src/__tests__/build/ enforces that. The slow-chromium-e2e and slow-firefox-e2e jobs gate the same way with the same key.

Limitations and known gaps

Vetrix runner sandbox. The job runs npm ci --legacy-peer-deps and npm run build-storybook inside the node:22-bookworm image. Storybook 8.4.x is pinned (see .storybook/main.ts header) due to a webpack-cache crash in 8.6.x against Next 15's bundled webpack; the CI pin matches the local-dev pin.
No story-level snapshot exclusions yet. Stories that render non-deterministic content (timestamps, animations) need parameters: { chromatic: { disableSnapshot: true } } set on the story export. The current story library does not exercise this yet — add the parameter when the first non-deterministic story lands.
No theme-matrix snapshots yet. Chromatic supports rendering a story under multiple Storybook globals (dark mode, RTL, density) via the modes parameter. We have not configured modes because the app does not yet ship a dark-mode token set; revisit when it does.
No accessibility gate inside Chromatic. Chromatic offers a paid axe-core integration; the team's accessibility coverage continues to live in the npm run test:a11y gate against the live app, and in the Playwright @a11y project. The Storybook harness does not duplicate that coverage.

Troubleshooting

Symptom	Likely cause	Resolution
Job log: "CHROMATIC_PROJECT_TOKEN not set on runner"	Operator setup step 2 not yet complete.	Inject the runner-side env var per Operator setup step 2.
Job log: "Invalid project token"	Token was rotated in Chromatic.	Copy the new token from the Chromatic project page and update the runner env.
Chromatic UI shows "0 stories captured"	`npm run build-storybook` produced an empty bundle.	Verify `.storybook/main.ts` `stories:` glob picks up the story file; check the build log for Storybook compilation errors.
PR has no Chromatic status check	The Vetrix runner did not schedule the job (CI manifest not found).	Confirm `vetrix-ci.yml` is at the repo root and the runner's pipeline-resolution log mentions it.
Job log: "npm ci" fails with peer-dependency error	Missing `--legacy-peer-deps` flag (or `package-lock.json` drifted under a different install mode).	The CI job already passes `--legacy-peer-deps`; if the failure persists, run `npm install --legacy-peer-deps` locally to refresh the lockfile and commit.

Two of the three files below live in the vetrix-frontend application repository (the repo this harness actually runs in), not in this documentation repository, so they cannot be expressed as in-repo relative links here; the pipeline-reference doc lives in this repository and is linked directly:

../../user-docs/cicd/pipeline-reference.md — Vetrix CI YAML schema reference.
.storybook/main.ts (in vetrix-frontend) — Storybook configuration and version-pin rationale.
README.md (in vetrix-frontend) — Storybook dev/build commands and high-level repo layout.