CasePilot - AI Test Case Generator for Azure DevOps

Writing test cases by hand is slow, inconsistent, and often incomplete. CasePilot reads your User Stories -- title, description, acceptance criteria, and team discussion -- and produces structured, categorized, automation-ready Test Case work items directly in Azure DevOps. No copy-pasting, no context switching, no forgotten edge cases.

Features

Three-Pass AI Quality (Worker + Judge + Optimizer)

Every generation runs through three AI passes. The Worker creates initial test cases. The Judge reviews output, fills coverage gaps, removes duplicates. The Optimizer detects shared preconditions and overlapping tests, suggests grouping. Quality scores 8-9/10 consistently -- significantly higher than single-pass tools.

Built on ISTQB Test Design Techniques

Generation is guided by formal ISTQB techniques: Equivalence Partitioning, Boundary Value Analysis, Decision Table Testing, State Transition Testing, and Error Guessing. These are applied internally for systematic coverage -- the output reads like test cases written by an experienced QA engineer.

BDD / Gherkin Output Mode

Switch between traditional test case steps and BDD/Gherkin scenarios (Given/When/Then). Gherkin output uses declarative style, supports Scenario Outlines with Examples tables, and goes through the same two-pass AI quality review.

Full Automation Context

Each generated test case includes numbered steps with expected results, test data suggestions, preconditions, and optional automation hints. Test cases are created as native Azure DevOps Test Case work items with properly formatted steps -- ready for manual execution or automation scripting.

Project Knowledge

Configure your project's tech stack, UI component conventions, business rules, and testing standards in the Settings hub. This context is injected into every generation, producing test cases that match your project's terminology and patterns instead of generic output.

Wiki Integration

Link Azure DevOps Wiki pages as additional context. The generator pulls page content automatically, so domain knowledge documented in your Wiki feeds directly into test case generation without any manual effort.

Smart Categorization and Priority

Test cases are automatically categorized as positive, negative, or edge case and assigned a priority (1-Critical through 4-Low). This makes triage and test planning faster from the moment test cases are created.

Non-Overlapping Regeneration

Already have linked Test Cases for a User Story? The generator detects them and produces only new, non-overlapping test cases. Run it multiple times as requirements evolve without creating duplicates.

Export to CSV and JSON

Export generated test cases before or after saving them. Use CSV for spreadsheet review or JSON for integration with external tools and test management workflows.

Accessibility Auto-Layer (WCAG 2.2)

Automatically detects which WCAG 2.2 success criteria apply to your User Story based on UI patterns (forms, images, modals, navigation). When you select the Accessibility strategy, the AI generates targeted accessibility test cases for the detected criteria.

Requirement Drift Detection

When your User Story has linked Test Cases, CasePilot detects if the requirements have drifted -- acceptance criteria that no test covers, or test cases that no longer align with current requirements. Shows drift level, uncovered requirements, and stale TCs.

Mutation Score Grades

Each generated test case receives an A-F grade predicting its "bug-killing power" -- how effective it would be at catching real code mutations. Based on 8 quality signals: assertion specificity, boundary coverage, state verification, error paths, data variety, step count, precondition quality, and independence.

Framework-Specific Hints

Select your automation framework (Playwright, Selenium, or Cypress) and get framework-specific automation hints -- Locator API for Playwright, WebDriver for Selenium, cy.get() for Cypress.

6 Test Strategy Templates

Choose a strategy that fits your need: Comprehensive, Smoke Test, Regression, Security (OWASP Top 10), Performance, or Accessibility (WCAG 2.2).

Security Lens (OWASP Top 10)

Automatically detects OWASP Top 10 risks in your User Story. When Security strategy is selected, generates targeted security test cases for the identified risks.

Free Tier

Get started with 20 test cases per month at no cost and with no registration. Install the extension and start generating immediately.

How It Works

1. Open a User Story -- Navigate to any User Story work item in Azure Boards and open the Test Case Generator tab.

2. Configure (optional) -- Adjust generation settings: maximum test case count, categories to include, and whether to add automation hints. Attach Wiki pages for additional context.

3. Generate -- Click Generate Test Cases. The AI reads your User Story's title, description, acceptance criteria, and discussion comments, then runs the two-pass Worker + Judge pipeline.

4. Review and Save -- Preview the generated test cases, select the ones you want, and save them as linked Test Case work items in Azure DevOps with a single click.

Screenshots

Public API

CasePilot offers a REST API for integrating test case generation into any workflow — CI/CD pipelines, GitHub Issues, Linear, Notion, custom scripts, and more.

• Authentication: API key via X-API-Key header
• Endpoint: POST /api/v1/generate
• Documentation: API Docs
• OpenAPI Spec: openapi.json

API access requires a paid plan. Generate your API key from the extension settings.

Getting Started

1. Install -- Add the extension to your Azure DevOps organization from the Marketplace.

2. Configure Project Knowledge (optional) -- Go to Project Settings > AI Test Case Settings and enter your tech stack, UI components, business rules, and testing conventions. Add Wiki page paths if you want Wiki content included in generation context.

3. Generate -- Open any User Story, switch to the Test Case Generator tab, and click Generate Test Cases. Review the results, select the test cases you want to keep, and click Save to Azure DevOps.

That's it. No API keys to manage, no external accounts to create. The extension works out of the box on the free tier.

Pricing

Annual billing available with 17% savings. All plans include unlimited users. See full pricing.

Also from IK Lab

BugPilot — AI Bug Report Enhancer — Transform vague bug reports into structured, actionable reports with AI-generated repro steps, severity suggestions, and missing info detection.

Support

Questions or issues? Email support@iklab.dev.