TestPilot AI

AI-powered test case generation, automation scaffolding, self-healing, and JIRA integration — right inside VS Code.

Features

Sidebar UI

TestPilot AI adds a dedicated panel in the VS Code Activity Bar with a React-based interface:

• Input tabs — switch between URL, Source code, Guidelines (acceptance criteria / user stories), and JIRA ticket input
• Add Related Files — provide a path to an existing spec or test file for context-aware generation
• Numbered generate buttons — step through Test Cases → Selectors → Mock Data → Test Files
• Self-Heal — paste a failing test error and click Heal to auto-fix it in agent mode
• Progress bar — real-time feedback while generation is running
• Output panel — view generated test cases and JIRA ticket details in separate tabs in an editor panel

AI Provider

TestPilot AI supports three AI back-ends, configured via test-pilot.aiExtension (default: copilot):

Note: The sidebar generate buttons work in all three modes. The @test-pilot-qa Copilot Chat commands are only available when test-pilot.aiExtension is set to copilot.

Chat Participant — @test-pilot-qa

Available only when test-pilot.aiExtension is copilot. Use TestPilot AI directly in GitHub Copilot Chat:

All commands except /testcases run in agent mode so Copilot can write files directly to your workspace.

Multi-Stack Support

Choose your target framework at generation time:

• Default Web — Playwright + Cucumber (TypeScript)
• Default Mobile — WDIO + Cucumber (TypeScript)

Project Scaffolding

The Setup Test Framework button detects your workspace and scaffolds a complete, production-ready framework including:

• Playwright & WDIO driver adapters
• Page Object base classes (BaseActions, BaseSelectors, BaseIdentifiers)
• Cucumber config, tsconfig, and playwright.config.ts
• Allure reporting setup (web + mobile)
• Mobile mock server (json-server + step definitions)
• All required package.json scripts

JIRA Integration

Connect TestPilot AI to your Atlassian JIRA instance to fetch ticket details (summary, description, acceptance criteria) and use them as test generation input — no copy-pasting required. Any Confluence links within a ticket are automatically fetched and included as context.

Figma Integration

Attach a Figma API token to enrich JIRA stories with design context. When a JIRA ticket contains Figma links, TestPilot AI fetches the referenced designs and describes them using vision AI before generating test cases.

Requirements

• VS Code 1.103.0 or higher
• A TestPilot AI API key (see Getting Access below)
• GitHub Copilot Chat — required only when test-pilot.aiExtension is copilot (the default)

Getting Access

TestPilot AI requires an API key tied to your work email.

1. Open the sidebar — the sign-up screen appears on first launch.
2. Enter your work email and click Request Access.
3. You will receive an API key by slack.
4. Enter the key in the Authorize step and click Authorize.

Once authorised, TestPilot AI loads and your session persists across VS Code restarts. To sign out, use the account menu in the sidebar header.

Getting Started

1. Open the Sidebar

Click the TestPilot AI icon in the VS Code Activity Bar (left sidebar) and complete the sign-up flow above.

2. Generate Test Cases

Via the sidebar:

1. Select an input tab: URL, Source, Guidelines, or JIRA
2. Enter your input (paste a URL, source code, acceptance criteria, or a JIRA ticket ID)
3. Optionally add a related file path in the Add Related Files step
4. Click 1) Test Cases — test cases appear in the output panel

Via Copilot Chat (requires copilot mode):

@test-pilot-qa /testcases <your feature spec, source code, or URL>

You will be prompted to choose a test type: System Testing, E2E Testing, or Unit Testing. Attach files with the paperclip button for additional context.

3. Review & Approve Test Cases

Generated test cases appear in the Test Cases tab. Filter by section (Functional / Non-Functional / Security), select individual rows or click Approve All, then proceed to step 4.

4. Generate Test Files

Via the sidebar: Click 4) Action, Feature & Step

Select Default Web (Playwright + Cucumber) or Default Mobile (WDIO + Cucumber) when prompted. Copilot agent will create the Actions, Feature, and Step files in your workspace.

5. Generate Selectors

@test-pilot-qa /selectors <page URL or component description>

6. Generate Mock Data

@test-pilot-qa /mockdata

Runs against the currently approved test cases.

7. Self-Heal Failing Tests

Via the sidebar:

1. Paste the test error output in the Self Heal section
2. Click Heal

Via Copilot Chat:

@test-pilot-qa /heal <paste your error here>

8. Set Up a New Test Project

Click Setup Test Framework in the sidebar. Select a target folder and choose Web, Mobile, or Both — TestPilot AI will scaffold the full framework and run yarn install automatically.

JIRA Configuration

Open VS Code Settings (Ctrl+, / Cmd+,) and search test-pilot:

Figma Configuration

Commands

Extension Settings

Known Issues

Please report issues on the GitHub repository.

Release Notes

0.0.2

• API key authentication — request access via email; session persists across restarts
• Multi-AI provider support — switch between Copilot, Claude Code, or custom clipboard fallback via test-pilot.aiExtension
• Figma integration — JIRA stories auto-enriched with Figma design context
• Confluence integration — linked Confluence pages automatically fetched as test generation context
• Test type picker for /testcases — choose System Testing, E2E Testing, or Unit Testing
• Improved error handling across all services with structured console logging
• Output panel now uses tabbed layout (Test Cases | JIRA)

0.0.1

Initial release:

• Sidebar UI with URL / Source / Guidelines / JIRA input tabs
• One-click generation of test cases, selectors, mock data, and test files
• Self-heal section for auto-fixing failing tests
• Output panel with test case review and JIRA detail tabs
• @test-pilot-qa chat participant with /testcases, /lazy-automate, /selectors, /mockdata, /codegen, and /heal commands
• Multi-stack support: Playwright + Cucumber (Web) and WDIO + Cucumber (Mobile)
• Project scaffolding via Setup Test Framework button with full framework structure
• JIRA integration with configurable credentials

Enjoy testing with TestPilot AI!