Visual Regression Testing

Automate visual regression testing for your web applications. Compare feature branch changes against main branch baselines automatically.

Demo

images/visual-regression-testing-vscode-extension.mp4

Features

• 📱 Side panel interface - Dedicated panel in the Activity Bar for easy access to all testing features
• 💾 Saved URL sets - Save and manage URL sets for repeated testing workflows
• ⚠️ Branch protection - Visual warning when on main branch to prevent accidental test runs
• 📋 Configuration display - View current settings at a glance in the panel
• 🎯 Automatic baseline comparison - Captures main branch screenshots and compares with your feature branch
• 🔄 Smart branch switching - Handles all Git operations automatically
• 🚀 Zero configuration - Works with your existing Playwright setup
• 📊 HTML reports - Side-by-side comparison of changes
• 🌍 Environment variables - Pass custom variables for auth bypass and more
• ⚡ Multiple URL testing - Test several routes in one run
• 📍 Status bar integration - Quick access to tests and reports from the status bar (optional)

Installation

1. Install from VS Code Marketplace or download the .vsix file
2. Ensure Playwright is installed: npm install -D @playwright/test && npx playwright install

📚 New to this extension? Check out the Complete Getting Started Guide for a step-by-step tutorial.

Quick Start

1. Create Test File

Create tests/visual/pages.spec.ts:

import { test, expect } from '@playwright/test';

// Get URLs from environment (comma-separated for multiple URLs)
const testUrls = process.env.TEST_URLS 
  ? process.env.TEST_URLS.split(',')
  : [process.env.TEST_URL || 'http://localhost:3000/'\]\;

// Generate a test for each URL
for (const testUrl of testUrls) {
  test(\`visual test for \${testUrl}\`, async ({ page }) => {
    await page.goto(testUrl);
    await page.waitForLoadState('networkidle');

    // Generate unique filename from URL path
    const urlPath = new URL(testUrl).pathname;
    const filename = urlPath
      .replace(/^\//, '')
      .replaceAll('/', '-')
      .replaceAll(/[^a-zA-Z0-9-_]/g, '_')
      || 'homepage';

    await expect(page).toHaveScreenshot(\`\${filename}.png\`, { fullPage: true });
  });
}

2. Configure Settings

Add to .vscode/settings.json:

{
  "visualRegression.testPath": "tests/visual",
  "visualRegression.serverStartCommand": "npm run dev",
  "visualRegression.serverPort": 3000,
  "visualRegression.mainBranch": "main",
  "visualRegression.testImportPath": "../path-to-custom-fixtures",
  "visualRegression.waitForSelector": "selector-rule-to-show-page-has-loaded",
  "visualRegression.environmentVariables": {
    "NEXT_PUBLIC_PLAYWRIGHT": "true"
  }
}

3. Run Test

Option 1: Using Side Panel ⭐ Recommended

1. Click the camera icon (📷) in the Activity Bar (left sidebar)
2. Enter URL paths in the text field (one per line):

   /
   /about
   /contact
   

3. Click "▶ Run Test"
4. The panel shows your current branch, configuration, and test progress
5. Save frequently-used URL sets for quick re-testing

Option 2: Using Command Palette

1. Open Command Palette (`Cmd+Shift+P`)
2. Type "Visual Regression: Run Test"
3. Enter URL paths (comma-separated for multiple): `/,/unauthorised,/access-denied`

Option 3: Using Status Bar

1. Click the "Visual Tests" item in the status bar (bottom left)
2. Select "Run Test" or "Show Report" from the menu

How It Works

1. Clears existing snapshots
2. Switches to main branch and captures baseline screenshots
3. Returns to your feature branch
4. Compares new screenshots against baseline
5. Shows HTML report if differences are found

Configuration

Environment Variables

Pass variables to bypass auth or enable mocking:

{
  "visualRegression.environmentVariables": {
    "NEXT_PUBLIC_PLAYWRIGHT": "true",
    "API_MOCKING": "enabled"
  }
}

Auth Bypass Example (Next.js)

In your middleware:

export function middleware(request: NextRequest) {
  if (process.env.NEXT_PUBLIC_PLAYWRIGHT === 'true') {
    return NextResponse.next();
  }
  // Normal auth flow...
}

Troubleshooting

Test file not found

Create `tests/visual/pages.spec.ts` with the template above.

Playwright not installed

npm install -D @playwright/test && npx playwright install

Not a Git repository

git init

Auth redirects

Add environment variables to bypass auth (see Auth Bypass Example above).

Server timeout

Increase `serverStartupTime` in settings (e.g., `10000` for 10 seconds).

Port already in use

Stop your dev server before running tests, or change `serverPort` in settings.

No visual differences detected

Expected if your changes don't affect visual appearance.

Not seeing notifications

VS Code notifications appear in the bottom right corner. If you missed them:

1. Click the bell icon (🔔) in the bottom right status bar
2. Or check View → Notifications to see recent notifications
3. Alternatively, check the Output panel for detailed logs

Disable Playwright Test UI

1. Open Settings (`Cmd+,`)
2. Search "Playwright"
3. Uncheck "Playwright: Show Test Explorer"

Hide/Show Status Bar

The extension adds a "Visual Tests" item to the status bar for quick access. To hide it:

1. Open Settings (`Cmd+,`)
2. Search "Visual Regression: Show Status Bar"
3. Uncheck to hide, or check to show
4. Changes apply immediately - no reload required

View logs

1. Open Output panel: `View → Output`
2. Select "Visual Regression Testing"

Using the Side Panel

Accessing the Panel

Click the camera icon (📷) in the Activity Bar to open the Visual Regression Testing panel.

Panel Features

Branch Warning

• Shows a warning when you're on the main/base branch
• Prevents accidental test runs on the baseline branch

Configuration Display

• Current Branch
• Main Branch
• Server Port
• Test Path

URL Input

• Enter multiple URLs (one per line)
• URLs are automatically formatted
• Persists between sessions

Saved URL Sets

• Save frequently-used URL combinations with a name
• Quickly re-run saved test sets
• Edit/Load URLs from saved sets
• Delete sets you no longer need

Test Actions

• Run Test: Execute visual regression test with current URLs
• Show Report: Open the latest Playwright HTML report
• Running banner replaces buttons during test execution

Commands

All features are accessible via the side panel, command palette, or status bar:

• Run Test - Test one or more URLs
• Run Full Workflow - Complete test workflow
• Run Tests for Changed Pages - Test only modified pages
• Run Test for Single Page - Test a specific page
• Update Baselines - Update baseline screenshots
• Show Playwright Report - View latest test report
• Clean Snapshots - Remove temporary snapshots

Requirements

• Node.js 16+
• Git repository
• Playwright (`@playwright/test`)
• Dev server command

Licence

MIT

Contributions

Love this extension? Star us and buy me a coffee

Want to make this extension even more awesome? Send us your wish.

Hate how it is working? Raise an issue.