Testcase Creation Agent — VS Code Extension

Install this extension to get the Testcase Creation agent in your Copilot Chat. It scans API schemas and Azure DevOps User Stories, generates test case proposals, and pushes approved tests to ADO.

Installation

From VSIX (Internal Distribution)

Download the .vsix file from your team's shared location
In VS Code: Ctrl+Shift+P → Extensions: Install from VSIX...
Select the downloaded .vsix file
Reload VS Code

From Source

cd extension
npm install
npx vsce package
# Produces: testcase-creation-mcp-1.0.0.vsix

Prerequisites

Python 3.10+ installed on your machine

pip install the server package in your project:

pip install -e <path-to-this-repo>

Or install in your project's .venv:

.venv\Scripts\pip install -e <path-to-this-repo>

Usage

Once installed, use @Testcase Creation in the Copilot Chat panel:

@Testcase Creation scan the schema at ./specs/openapi.json
@Testcase Creation analyze coverage for User Story #12345 against suite 200 in plan 100
@Testcase Creation push all approved tests to User Story #12345

Configuration

Setting	Description	Default
`testcaseCreation.pythonPath`	Path to Python executable	Auto-detected from `.venv`
`testcaseCreation.adoPat`	Azure DevOps PAT	(empty — use `configure_credentials` tool instead)

Commands

Command	Description
`Testcase Creation: Start MCP Server`	Manually start the MCP server process
`Testcase Creation: Stop MCP Server`	Stop the running server
`Testcase Creation: Show Available Tools`	View all 27 tools with descriptions

Available Tools (27)

Core Tools

Tool	What it does
`scan_schema_and_generate_tests`	Scan OpenAPI/JSON Schema, generate test proposals
`push_approved_tests`	Push approved proposals to ADO
`generate_test_steps_excel`	Generate Excel with test steps
`list_pending_proposals`	List proposals awaiting review
`push_excel_to_ado`	Push from edited Excel to ADO
`analyze_schema_coverage`	Coverage gap analysis (ADO vs schema)
`generate_excel_from_schema`	Generate Excel from scan results
`generate_markdown_from_proposals`	Generate Markdown review doc
`push_markdown_to_ado`	Push from edited Markdown to ADO
`scan_user_story`	Parse ADO User Story -> test proposals
`analyze_us_coverage`	Requirement coverage vs ADO suite
`list_ado_suite_testcases`	List TCs in an ADO suite
`fetch_testcase_from_ado`	Fetch TC steps from ADO
`update_testcase_outcome_in_testplan`	Post pass/fail to ADO test plan
`generate_excel_from_ado`	Pull TCs from ADO into Excel
`backfill_ado_ids`	Write ADO IDs back to Excel
`generate_mismatch_report`	ADO vs Excel discrepancy report
`configure_credentials`	Set ADO PAT / tokens at runtime
`list_projects`	List registered knowledge library projects
`validate_credentials`	Check credential validity

Enhanced Tools

Tool	What it does
`generate_test_templates`	Pre-built CRUD/auth/pagination test patterns
`generate_ai_test_steps`	AI-powered test step generation with fallback
`generate_test_data`	Boundary/edge-case test data generation
`analyze_regression_impact`	Map schema changes to affected TCs
`detect_e2e_flows`	Detect E2E flows from API specs
`analyze_api_dependencies`	Build API dependency graph
`configure_ai_generation`	Set AI provider keys for step generation

How It Works

You (Copilot Chat)  ──►  Testcase Creation Agent  ──►  MCP Server (Python)
                                                            │
                                    ┌───────────────────────┼───────────────┐
                                    ▼                       ▼               ▼
                              Schema Files           Azure DevOps       Excel/Markdown
                              (OpenAPI, JSON)        (Test Cases)       (Review docs)

You ask the agent to scan a schema or user story
Agent generates test case proposals and presents them
You review and approve (mandatory human gate)
Agent pushes approved tests to ADO with full traceability

Troubleshooting

Issue	Solution
"MCP server failed to start"	Check Python path in settings, ensure package is installed
"No module named test_automation"	Run `pip install -e .` in the project root
"ADO operations failing"	Use `configure_credentials` to set your ADO PAT
Agent not appearing in chat	Reload window after installation (`Ctrl+Shift+P` → Reload Window)

OI ADO Testcase Creation MCP Agent

oi-automation

Testcase Creation Agent — VS Code Extension

Installation

From VSIX (Internal Distribution)

From Source

Prerequisites

Usage

Configuration

Commands

Available Tools (27)

Core Tools

Enhanced Tools

How It Works

Troubleshooting