Visual Regression Testing

Automate visual regression testing for your web applications. Compare feature branch changes against main branch baselines automatically.

Demo

images/visual-regression-testing-vscode-extension.mp4

Features

• 📱 Side panel interface - Dedicated panel in the Activity Bar for easy access to all testing features
• 💾 Saved URL sets - Save and manage URL sets for repeated testing workflows
• ⚠️ Smart warnings - Alerts for uncommitted changes and unsaved test set modifications before running tests
• 🔒 Branch protection - Visual warning when on main branch to prevent accidental test runs
• 📦 Persistent snapshot storage - Automatically saves baseline and comparison snapshots for later review
• 📋 Configuration display - View current settings at a glance in the panel
• 🎯 Automatic baseline comparison - Captures main branch screenshots and compares with your feature branch
• 🔄 Smart branch switching - Handles all Git operations automatically
• 🚀 Zero configuration - Works with your existing Playwright setup
• 📊 HTML reports - Side-by-side comparison of changes with interactive slider
• 🌍 Environment variables - Pass custom variables for auth bypass and more
• ⚡ Multiple URL testing - Test several routes in one run
• 📍 Status bar integration - Quick access to tests and reports from the status bar (optional)

Installation

1. Install from VS Code Marketplace or download the .vsix file
2. Ensure Playwright is installed: npm install -D @playwright/test && npx playwright install

📚 New to this extension? Check out the Complete Getting Started Guide for a step-by-step tutorial.

Quick Start

1. Create Test File

Create tests/visual/pages.spec.ts:

import { test, expect } from '@playwright/test';

// Get URLs from environment (comma-separated for multiple URLs)
const testUrls = process.env.TEST_URLS 
  ? process.env.TEST_URLS.split(',')
  : [process.env.TEST_URL || 'http://localhost:3000/'\]\;

// Generate a test for each URL
for (const testUrl of testUrls) {
  test(\`visual test for \${testUrl}\`, async ({ page }) => {
    await page.goto(testUrl);
    await page.waitForLoadState('networkidle');

    // Generate unique filename from URL path
    const urlPath = new URL(testUrl).pathname;
    const filename = urlPath
      .replace(/^\//, '')
      .replaceAll('/', '-')
      .replaceAll(/[^a-zA-Z0-9-_]/g, '_')
      || 'homepage';

    await expect(page).toHaveScreenshot(\`\${filename}.png\`, { fullPage: true });
  });
}

2. Configure Settings

Add to .vscode/settings.json:

{
  "visualRegression.testPath": "tests/visual",
  "visualRegression.serverStartCommand": "npm run dev",
  "visualRegression.serverPort": 3000,
  "visualRegression.mainBranch": "main",
  "visualRegression.testImportPath": "../path-to-custom-fixtures",
  "visualRegression.waitForSelector": "selector-rule-to-show-page-has-loaded",
  "visualRegression.environmentVariables": {
    "NEXT_PUBLIC_PLAYWRIGHT": "true"
  }
}

3. Run Test

Option 1: Using Side Panel ⭐ Recommended

1. Click the camera icon (📷) in the Activity Bar (left sidebar)
2. Enter URL paths in the text field (one per line):

   /
   /about
   /contact
   

3. Click "▶ Run Test"
4. The panel shows your current branch, configuration, and test progress
5. Save frequently-used URL sets for quick re-testing

Option 2: Using Command Palette

1. Open Command Palette (`Cmd+Shift+P`)
2. Type "Visual Regression: Run Test"
3. Enter URL paths (comma-separated for multiple): `/,/unauthorised,/access-denied`

Option 3: Using Status Bar

1. Click the "Visual Tests" item in the status bar (bottom left)
2. Select "Run Test" or "Show Report" from the menu

How It Works

1. Safety checks - Warns if you have uncommitted changes or unsaved test set modifications
2. Clears existing snapshots - Removes temporary snapshots from previous runs
3. Captures baseline - Switches to main branch and captures baseline screenshots
4. Saves baseline - Stores baseline snapshots to .visual-regression-snapshots/baseline/
5. Returns to feature branch - Switches back to your working branch
6. Runs comparison - Compares new screenshots against baseline
7. Saves comparison - Stores comparison results to .visual-regression-snapshots/comparison/
8. Shows HTML report - Opens interactive report if differences are found

Persistent Snapshot Storage

The most recent test results are automatically saved to .visual-regression-snapshots/ (gitignored):

• Baseline snapshots: .visual-regression-snapshots/baseline/

  • Contains the baseline screenshots from the main branch
  • Includes the .spec.ts-snapshots/ directory with expected images
  • Includes the Playwright HTML report

• Comparison snapshots: .visual-regression-snapshots/comparison/

  • Contains screenshots from your feature branch
  • Includes the .spec.ts-snapshots/ directory with expected images
  • Includes test-results/ with actual, expected, and diff images (when differences are found)
  • Includes the complete Playwright HTML report for interactive comparison

Previous test results are automatically cleaned up when you run a new test, keeping only the most recent baseline and comparison snapshots. You can open the saved playwright-report/index.html in any browser to review the comparison with the interactive slider.

Configuration

Environment Variables

Pass variables to bypass auth or enable mocking:

{
  "visualRegression.environmentVariables": {
    "NEXT_PUBLIC_PLAYWRIGHT": "true",
    "API_MOCKING": "enabled"
  }
}

Auth Bypass Example (Next.js)

In your middleware:

export function middleware(request: NextRequest) {
  if (process.env.NEXT_PUBLIC_PLAYWRIGHT === 'true') {
    return NextResponse.next();
  }
  // Normal auth flow...
}

Troubleshooting

Test file not found

Create `tests/visual/pages.spec.ts` with the template above.

Playwright not installed

npm install -D @playwright/test && npx playwright install

Not a Git repository

git init

Auth redirects

Add environment variables to bypass auth (see Auth Bypass Example above).

Server timeout

Increase `serverStartupTime` in settings (e.g., `10000` for 10 seconds).

Port already in use

Stop your dev server before running tests, or change `serverPort` in settings.

No visual differences detected

Expected if your changes don't affect visual appearance.

Not seeing notifications

VS Code notifications appear in the bottom right corner. If you missed them:

1. Click the bell icon (🔔) in the bottom right status bar
2. Or check View → Notifications to see recent notifications
3. Alternatively, check the Output panel for detailed logs

Disable Playwright Test UI

1. Open Settings (`Cmd+,`)
2. Search "Playwright"
3. Uncheck "Playwright: Show Test Explorer"

Hide/Show Status Bar

The extension adds a "Visual Tests" item to the status bar for quick access. To hide it:

1. Open Settings (`Cmd+,`)
2. Search "Visual Regression: Show Status Bar"
3. Uncheck to hide, or check to show
4. Changes apply immediately - no reload required

View logs

1. Open Output panel: `View → Output`
2. Select "Visual Regression Testing"

Reviewing saved snapshots

After running tests, you can review both baseline and comparison snapshots:

1. Navigate to .visual-regression-snapshots/ in your workspace
2. You'll find two directories:
   • baseline/ - Contains main branch screenshots and Playwright report
   • comparison/ - Contains feature branch screenshots, diff images, and Playwright report
3. Open the saved HTML report: Navigate to either directory and open playwright-report/index.html in your browser for the full interactive comparison with the slider
4. Browse images directly: Look in the comparison/test-results/ folder for actual, expected, and diff PNG files
5. Snapshot directories are automatically gitignored and cleaned up with each new test run

Using the Side Panel

Accessing the Panel

Click the camera icon (📷) in the Activity Bar to open the Visual Regression Testing panel.

Panel Features

Branch Warning

• Shows a warning when you're on the main/base branch
• Prevents accidental test runs on the baseline branch

Configuration Display

• Current Branch
• Main Branch
• Server Port
• Test Path

URL Input

• Enter multiple URLs (one per line)
• URLs are automatically formatted
• Persists between sessions

Saved URL Sets

• Save frequently-used URL combinations with a name
• Quickly re-run saved test sets
• Edit/Load URLs from saved sets
• Delete sets you no longer need
• Auto-save prompt: When running tests with unsaved changes to a loaded set, you'll be prompted to save

Safety Warnings

• Uncommitted changes: Warns if you have uncommitted changes (excluding page.spec.ts) before running tests
• Unsaved test set changes: Detects modifications to loaded test sets and offers to save before running
• Options: Save Changes, Run Anyway, or Cancel

Test Actions

• Run Test: Execute visual regression test with current URLs
• Show Report: Open the latest Playwright HTML report
• Running banner replaces buttons during test execution

Commands

All features are accessible via the side panel, command palette, or status bar:

• Run Test - Test one or more URLs
• Run Full Workflow - Complete test workflow
• Run Tests for Changed Pages - Test only modified pages
• Run Test for Single Page - Test a specific page
• Update Baselines - Update baseline screenshots
• Show Playwright Report - View latest test report
• Clean Snapshots - Remove temporary snapshots

Requirements

• Node.js 16+
• Git repository
• Playwright (`@playwright/test`)
• Dev server command

Licence

MIT

Contributions

Love this extension? Star us and buy me a coffee

Want to make this extension even more awesome? Send us your wish.

Hate how it is working? Raise an issue.