🤖 RegressBot - Intelligent Test Automation for VS Code

🚀 AI-powered Playwright test generation, design comparison, and user story validation — all seamlessly integrated with GitHub Copilot Chat

✨ Key Features

📋 Prerequisites

• ✅ VS Code 1.85.0 or higher
• ✅ GitHub Copilot extension installed
• ✅ Node.js 18+ and npm
• ✅ Modern browser (Chrome, Firefox, Edge, Safari)

🚀 Installation & Setup

📦 From VS Code Marketplace

1. Open VS Code and press Ctrl+Shift+X (Extensions)
2. Search for "RegressBot"
3. Click Install
4. Done! 🎉

🔧 Manual Installation (Development)

# Clone the repository
git clone <repo-url>
cd RegresBot

# Install dependencies
npm install

# Compile TypeScript
npm run compile

# Launch in development mode
Press F5

💡 How It Works

3 Simple Steps to Automate Your Testing

┌─────────────────────────────────────────────────────────────┐
│  1️⃣  Attach Centralized Prompt to Copilot Chat     │
│  2️⃣  Enter Your Command (explore, compare, validate)│
│  3️⃣  Let AI Handle the Rest!                        │
└─────────────────────────────────────────────────────────────┘

🎯 Usage Guide

🔧 Step 1: Initial Setup

Open Command Palette (Ctrl+Shift+P) and run:

RegresBot: Attach Centralized Prompt to Copilot Chat

✅ What this does:

• 📄 Opens the centralized routing prompt file
• 🎯 Click the 📎 attach button in Copilot Chat
• ⚙️ Go to Copilot Chat → Tools → enable Playwright
• 🔄 You may see a reload notification — click Reload to complete

🚀 Step 2: Use Commands in Copilot Chat

🔍 Command 1: Explore URL and Generate Tests

Syntax:

explore https://example.com

What happens:

• 🌐 Navigates the website module by module
• 🦾 Generates comprehensive Playwright test files automatically
• 💾 Saves to tests/{module-name}.spec.ts
• 📈 Provides 80%+ test coverage per module
• ✅ Executes tests to verify they work

Live Example:

explore https://angular.io

Sample Output:

// tests/authentication.spec.ts
test('should login successfully', async ({ page }) => {
  await page.goto('https://example.com/login');
  await page.getByLabel('Email').fill('user@example.com');
  await page.getByRole('button', { name: 'Login' }).click();
  await expect(page).toHaveURL(/.*dashboard/);
});

🎨 Command 2: Compare Design with Figma

Syntax:

compare design https://example.com

Required Inputs: | Input | Example | Purpose | |-------|---------|---------| | 🌐 URL | https://myapp.com | Live website to compare | | 📁 Figma Path | ./Figma/HomePage.png | Reference design file | | 🗺️ Process Flow | Login > Home > Settings | User journey to capture |

What happens:

• 🎬 Navigates through your process flow
• 📸 Captures live website screenshots at each step
• 🔍 Compares against Figma reference designs
• 📊 Generates HTML comparison report with visual diffs
• 💡 Provides actionable improvement recommendations

Example:

compare design https://myapp.com
Figma Path: ./Figma/HomePage.png
Process: Login > Home Page 

Report Output:

• ✨ Overall match percentage
• 📖 Category scores (layout, typography, colors, spacing)
• 🖼️ Side-by-side image comparison
• 🎯 Annotated diff highlighting differences
• 🚀 Actionable improvement recommendations

✅ Command 3: Validate User Stories

Syntax:

validate https://example.com

Required Inputs: | Input | Format | Details | |-------|--------|---------| | 🌐 URL | https://myapp.com | Website to validate against | | 📝 User Stories | .txt file | Plain text with user stories |

What happens:

• 📖 Parses user stories from your .txt file
• 🦾 Generates Playwright test cases automatically
• 🕷️ Crawls website to identify correct selectors
• ▶️ Executes all tests automatically in headed browser
• 📋 Creates validation report with pass/fail status
• 🔍 Provides detailed explanations for failures
• 💡 Generates recommendations for fixes

Example:

validate https://myapp.com
(Attach user-stories.txt to Copilot Chat)

Report Includes:

• ✅ Pass/fail summary per story
• 📊 Overall validation percentage
• 🔴 Failed test explanations
• 🚀 Fix recommendations

📁 Project Structure

After using RegresBot, your workspace will have:

📦 your-project/
├── 📂 .github/
│   └── 📂 prompts/
│       ├── centralized_tool.prompt.md ⚙️
│       ├── generate_tests.prompt.md 🦾
│       ├── compare_design.prompt.md 🎨
│       └── validate_user_story_tests.prompt.md ✅
│
├── 📂 tests/ 🦾
│   ├── module1.spec.ts
│   ├── module2.spec.ts
│   └── user-story.spec.ts
│
├── 📂 Figma/ 🎨
│   └── reference-designs.png
│
├── 📂 test-results/ 📊
│   ├── 📂 design-comparison/
│   │   ├── comparison-report.html 📈
│   │   ├── live-screenshot.png 📸
│   │   └── diff.png 🔍
│   └── VALIDATION_REPORT.md 📋
│
├── package.json 📦
└── playwright.config.ts ⚙️

⚙️ Configuration

Customize RegresBot via VS Code settings (settings.json):

{
  "regresbot.testDirectory": "tests",           // 🦾 Where to save test files
  "regresbot.figmaDirectory": "Figma",          // 🎨 Figma designs location
  "regresbot.reportDirectory": "test-results",  // 📊 Report output directory
  "regresbot.coverageThreshold": 80,             // 📈 Minimum coverage %
  "regresbot.browserType": "chromium"            // 🌐 Browser type (chromium|firefox|webkit)
}

📖 Available Commands

🔧 Detailed Workflows

🔍 Explore URL Workflow

1️⃣  AI navigates to URL
   ↓
2️⃣  Identifies all modules/features
   ↓
3️⃣  Explores each module comprehensively
   ↓
4️⃣  Generates test file per module
   ↓
5️⃣  Executes tests to verify
   ↓
6️⃣  Provides module-wise coverage reports 📊

🎨 Compare Design Workflow

1️⃣  Collects URL, Figma image, process flow
   ↓
2️⃣  Navigates & captures live website 📸
   ↓
3️⃣  Performs visual comparison using AI 🤖
   ↓
4️⃣  Calculates similarity scores 📈
   ↓
5️⃣  Generates annotated diff images 🖼️
   ↓
6️⃣  Creates HTML report with recommendations 📋

✅ Validate User Stories Workflow

1️⃣  Parses user stories from .txt file
   ↓
2️⃣  Crawls website for selectors 🕷️
   ↓
3️⃣  Generates Playwright test file 🦾
   ↓
4️⃣  Opens headed browser for visual feedback 🌐
   ↓
5️⃣  Handles login if required 🔐
   ↓
6️⃣  Executes all test scenarios ▶️
   ↓
7️⃣  Generates validation report 📊

🎨 Example Outputs

🦾 Generated Test Files

// tests/authentication.spec.ts
import { test, expect } from '@playwright/test';

test.describe('Authentication - Comprehensive Test Suite', () => {
  test('should login successfully with valid credentials', async ({ page }) => {
    // 🌐 Navigate to login page
    await page.goto('https://example.com/login');
    
    // ✏️ Enter credentials
    await page.getByLabel('Email').fill('user@example.com');
    await page.getByLabel('Password').fill('password123');
    
    // 👆 Click login button
    await page.getByRole('button', { name: 'Login' }).click();
    
    // ✅ Verify redirect to dashboard
    await expect(page).toHaveURL(/.*dashboard/);
  });
});

📊 Comparison Reports

HTML Dashboard includes:

• ✨ Overall match percentage
• 📖 Category scores (layout, typography, colors, spacing)
• 🖼️ Side-by-side image comparison
• 🔍 Annotated diff highlighting differences
• 💡 Actionable improvement recommendations
• 📈 Detailed analytics per section

📋 Validation Reports

Markdown reports with:

• 📊 Pass/fail summary by story
• ✅ Detailed results per user story
• 🔴 Failed test explanations
• 🚀 Recommendations for fixes
• 📈 Test execution time metrics
• 🌍 Browser compatibility notes

📄 License

📋 MIT License - see LICENSE file for details

⭐ Show Your Support

If this project helped you, give it a ⭐ star!

Your support helps us improve RegresBot and reach more developers! 🚀

Made with ❤️ for the testing community