API Test Execution MCP

Run data-driven API tests directly from VS Code Copilot Chat — GraphQL & REST, with dashboards, ADO sync, and email reports.

What This Extension Does

This extension registers an MCP (Model Context Protocol) server that exposes your API Automation Framework's test execution tools to GitHub Copilot Chat. Once installed, you get a "Testcase Execution" agent in Copilot Chat that can:

Execute tests — Run all, smoke, regression, or specific API tests from Excel or ADO
Generate dashboards — Rich HTML reports with charts, filters, and expandable details
Sync to ADO — Push pass/fail results back to Azure DevOps test plans
Send email reports — Dashboard + screenshot via SMTP
Full pipeline — Execute → Report → ADO Sync → Email in one command

Quick Start

Prerequisites

Python 3.10+ installed and on PATH
GitHub Copilot Chat extension installed
A clone of the API Automation repository

Installation

Install the .vsix:
- Command line: code --install-extension oi-api-testexec-mcp-0.3.0.vsix
- VS Code UI: Extensions panel → ··· menu → "Install from VSIX…"
Open the API Automation workspace folder in VS Code
The extension auto-configures:
- Creates .venv and installs dependencies (if needed)
- Registers the MCP server in .vscode/mcp.json
- Copies the agent definition to .github/agents/
If prompted, run: Ctrl+Shift+P → "API Test Execution: Setup Environment"

Using the Agent

Open Copilot Chat (Ctrl+Shift+I)
Select the "Testcase Execution" agent from the agent picker
Ask it to run tests:

Running Tests — Excel vs ADO, Scope Selection

The tool supports two data sources (Excel and ADO) and three scopes (smoke, regression, all).

From Excel (project or custom workbook)

What you say in chat	action	scope	What happens
"Run all tests for KAAS"	`run_all`	`all`	Runs every enabled test in the project's Excel
"Run smoke tests for KAT"	`run_smoke`	`smoke`	Runs only `suite=smoke` (or `both`) tests
"Run regression for KAAS"	`run_regression`	`regression`	Runs only `suite=regression` (or `both`) tests
"Run all tests from C:\my\tests.xlsx"	`run_all`	`all`	Runs from a custom Excel path

How scope works: Each test row in the Excel TestCases sheet has a suite column:

smoke — included only in smoke runs
regression — included only in regression runs
both (default) — included in all runs

From ADO Suite

What you say in chat	action	scope	What happens
"Run full ADO suite 2881718"	`run_from_ado`	`all`	Pulls all TCs from ADO suite, runs matching Excel rows
"Run smoke from ADO suite 2881718"	`run_from_ado`	`smoke`	Same, but only `suite=smoke`/`both` tests execute
"Run regression from ADO suite 2881718"	`run_from_ado`	`regression`	Same, but only `suite=regression`/`both` tests execute

Project Selection

Use project to select a registered project (resolves Excel, auth config, environment automatically):

Run smoke tests for project kaas
Run regression from ADO suite 2881718 for project kat

Or pass a custom Excel path:

Run all tests from excel_path C:\Users\me\MyTests.xlsx

Note: Either project or excel_path must be provided. The tool will not guess a default workbook.

Available Tools

Tool	Description
`data_driven_tool`	Core test runner — supports `run_all`, `run_smoke`, `run_regression`, `run_api`, `run_from_ado`, `list_apis`, `list_tests`, `get_summary`, `generate_dashboard`, `send_email`, `run_full_pipeline`
`backfill_excel_descriptions`	Fill empty description cells in Excel test data with human-readable text (supports dry-run mode)
`configure_credentials`	Set Bearer Token, Auth Cookie, ADO PAT, SetVersionId, or custom headers at runtime (in-memory only, never saved to disk)
`validate_credentials`	Check if current token/cookie are expired before running tests
`list_projects`	List all registered projects in the knowledge library
`list_ado_suite_testcases`	List test cases in an ADO Test Plan suite
`fetch_testcase_from_ado`	Read test case steps from ADO
`generate_excel_from_ado`	Pull TCs from ADO suite → generate Excel test data file
`backfill_ado_ids`	Sync ADO work item IDs into Excel test data
`update_testcase_outcome_in_testplan`	Push pass/fail results to ADO test plans

`data_driven_tool` Parameters

Parameter	Type	Description
`action`	string	One of: `run_all`, `run_smoke`, `run_regression`, `run_api`, `run_from_ado`, `list_apis`, `list_tests`, `get_summary`, `get_cached`, `generate_dashboard`, `send_email`, `run_full_pipeline`
`params.project`	string	Knowledge library project ID (e.g. `kaas`, `kat`) — resolves Excel, auth, and env
`params.excel_path`	string	Full path to an Excel workbook (overrides project)
`params.scope`	string	`all` (default), `smoke`, or `regression` — filters tests by suite column
`params.bearer_token`	string	Pre-supplied JWT token (skips OAuth2)
`params.auth_cookie`	string	Azure EasyAuth cookie
`params.api_id`	string	Required for `run_api` — which API to run
`params.plan_id`	int	ADO Test Plan ID (for `run_from_ado`)
`params.suite_id`	int	ADO Test Suite ID (for `run_from_ado`)
`params.update_ado`	bool	Auto-push pass/fail outcomes to ADO after `run_from_ado`

Extension Commands

Command	Description
`API Test Execution: Setup Environment`	Create `.venv`, install deps, register MCP server
`API Test Execution: Check Status`	Verify Python, dependencies, MCP config, and agent definition
`API Test Execution: Restart Server`	Re-register MCP config after changes

Configuration

Setting	Default	Description
`apiTestExecution.pythonPath`	(auto-detect)	Path to Python executable
`apiTestExecution.projectRoot`	(auto-detect)	Path to API Automation project root

Supported API Types

GraphQL — Auto-detected from Excel (no http_method column)
REST — Auto-detected from Excel (http_method column present)
Mixed — Both in the same project via the knowledge library

Features

Scope filtering — Run smoke, regression, or all tests from any data source
Retry with exponential backoff — Transient connection errors (timeout, reset) are retried automatically
Circuit breaker — APIs that fail 3 times consecutively are skipped for 30 seconds
Pre-flight credential validation — JWT tokens and cookies are checked for expiry before execution
Report deduplication — Dashboard and mismatch reports never overwrite previous files
Multi-project support — Switch between KAAS, KAT, or custom projects via project parameter
Custom headers — Inject extra headers at runtime via configure_credentials

Requirements

Dependency	Version
VS Code	≥ 1.99.0
Python	≥ 3.10
GitHub Copilot Chat	Latest
API Automation repo	Cloned locally

OI API TestExec MCP Agent

api-automation-team

API Test Execution MCP

What This Extension Does

Quick Start

Prerequisites

Installation

Using the Agent

Running Tests — Excel vs ADO, Scope Selection

From Excel (project or custom workbook)

From ADO Suite

Project Selection

Available Tools

`data_driven_tool` Parameters

Extension Commands

Configuration

Supported API Types

Features

Requirements

OI API TestExec MCP Agent

api-automation-team

API Test Execution MCP

What This Extension Does

Quick Start

Prerequisites

Installation

Using the Agent

Running Tests — Excel vs ADO, Scope Selection

From Excel (project or custom workbook)

From ADO Suite

Project Selection

Available Tools

data_driven_tool Parameters

Extension Commands

Configuration

Supported API Types

Features

Requirements

`data_driven_tool` Parameters