BDD Pilot

Run Reqnroll, SpecFlow, and Cucumber-style BDD tests from VS Code or Cursor — without hand-building dotnet test --filter strings or digging through raw console output.

BDD Pilot is the execution layer for .NET Gherkin projects: discover scenarios in a domain or @tag tree, run from the sidebar, native Test Explorer, or CodeLens, see pass/fail on every row (including Scenario Outline examples), and get actionable diagnostics when builds, NuGet, Playwright, or step bindings break.

Stable v1.0 — environment-aware runs (STAGE), sanitized logs, AI-ready failure context for Copilot/Cursor, and post-run feedback (error snippets in hover + optional summary toast). Framework-agnostic: any suite that runs through dotnet test (API, Playwright, UI, etc.).

Reqnroll on VS Code: the official Reqnroll extension targets Visual Studio 2022/2026. For VS Code / Cursor, use BDD Pilot to run tests and BDD Guardian to navigate step bindings — install both.

BDD extension family

BDD Pilot focuses on running tests. For navigation and step bindings, use BDD Guardian — they complement each other and can be installed side by side:

Pilot exposes a read-only Extension API (PilotRunApiV1) for in-process consumers such as Jarvis. TRX-on-disk remains the fallback when the API is absent.

Guardian answers “where is this step implemented?” — Pilot answers “run this scenario and show me what failed.”

Gallery

Cockpit frames (signals TE does not prioritize). Captures from Pilot on samples/minimal-bdd.

Tree cockpit — Gherkin tree + summary row (Last run pass/fail) and outline outcomes:

STAGE hub — status bar chip Pilot · STAGE · mode · project (here: test · parallel · MinimalBdd.Tests):

Run story — dashboard history, flaky scenarios, and diagnostics (mapping signals like not_in_trx / unmapped appear in the tree after scoped runs — see Cockpit signals):

More detail (editor CodeLens + Output diagnostics):

Install

• VS Code Marketplace: search BDD Pilot (publisher anghelll), or run:

  ext install anghelll.bdd-pilot
  

• Manual / pre-release: download the .vsix from GitHub Releases → Extensions → … → Install from VSIX…
• Try the sample: open samples/minimal-bdd/ as the workspace after installing.

Who it's for

Features

Pilot is a Gherkin cockpit alongside Test Explorer, not a TE replacement — domain/@tag tree, STAGE hub, TRX↔scenario mapping, and run history that TE does not prioritize.

Pilot vs Test Explorer

Use both: discover/run unit tests in TE; drive Reqnroll/SpecFlow scenarios from Pilot.

In pictures: Gallery — tree + summary · STAGE hub · dashboard run story.

Discovery & run

• Native Test Explorer (TestController): Run and Debug profiles with results; follows bddPilot.tree.groupBy (domain or @tag); descriptions mirror BDD tree settings (tree.durationDisplay, tree.tagDisplay) and locale for outcomes/roll-ups.
• BDD Pilot side view: Domain → Feature → Scenario tree from .feature files, with tag badges. Domain grouping uses a Feature/ or Features/ folder.
• Pilot summary row at the top of the tree — last run status (3 passed, Running…); dynamic icon while tests run (loading~spin) or debug (debug-alt); live progress in the description during runs (7/19 · 2 failed); post-run diagnostic chip (top actionable hint from the last run) with severity icon (warning / info); filter chip when tree search is active; unmapped chip after a scoped run when scenarios did not match TRX (N unmapped — Show Unmapped, command BDD Pilot: Show Unmapped Scenarios). Click opens dashboard (or the unmapped QuickPick when that chip is shown). Activity bar BDD Pilot icon shows a badge during active runs. Toolbar Dashboard icon ($(graph)) opens the same panel.
• Tree toolbar — Run · Search · Dashboard · Refresh · GroupBy · More (…) overflow for Re-run Failed and Execution Profiles; Debug inline on rows (bddPilot.debugNode); while running, Cancel replaces Run at the front of the toolbar and the summary row click cancels.
• Tree display mode (bddPilot.tree.displayMode): detailed (roll-ups on folders, default) or compact (less duplicate roll-ups; outcomes emphasized on leaves).
• CodeLens on Feature, Scenario, and Scenario Outline example rows (Run / Debug).
• One-click run: domain, feature, scenario, tag, or Scenario Outline row — the correct dotnet test --filter is built automatically.
  • Feature → FullyQualifiedName~<Feature>Feature
  • Scenario → FullyQualifiedName~<Feature>Feature.<Scenario>
  • Outline row → DisplayName~parameter: %22…%22, value: %22…%22 (single Theory row)
  • Tag → Category=<tag>
• Tree search to filter by name, tag, or path. Active filter shows on the summary row; Run filtered replaces Run All while filtered. Use @tag for tag-only matching. Persists per workspace. Ctrl+F highlights in the list only — Pilot Search controls run scope. Test Explorer does not inherit the tree filter.
• Re-run failed from the last run's filter.
• Saved execution profiles for common filters.

Environment & execution

• UI language (bddPilot.locale: auto | en | es) — status bar, dashboard, CodeLens, palette, and dialogs follow VS Code UI language when set to auto.
• Execution hub (status bar) — compact branded chip $(beaker) Pilot with STAGE, mode, and project (bddPilot.statusBar.display: compact default, detailed for legacy four items). Click to open unified settings QuickPick; while a run is active, Cancel is the first row.
• Environment (STAGE) (dev/test/stg/prod) — sets STAGE for the run; stg/prod require confirmation.
• Parallelism mode (debug/parallel/ci) passed as xUnit RunSettings, so the project's xunit.runner.json is never mutated on disk.
• Reliable execution: progress UI, cancellation, and live streaming to the BDD Pilot output channel.
• Copy effective command — palette copies the exact dotnet test … from the last Pilot run or debug (filter + run flags) for CI or a terminal.
• Debug launches dotnet test under the .NET debugger (coreclr).

Results & diagnostics

• TRX + Cucumber JSON: scenarios decorated with pass / fail / skip and duration.
• Tree mapping report: after a scoped run, Output shows mapped/inScope counts; when some scenarios lack a TRX match (not_in_trx), lists them (capped) and the palette command Show Unmapped Scenarios opens a QuickPick to jump to the .feature line. The same Output also lists TRX rows unused by any Gherkin leaf (capped) and ambiguous matches (first TRX row applied) — mapping honesty, not a generic unit-test runner.
• Webview dashboard: run history (with Scope per run, e.g. All tests / @tag), totals, enriched flaky scenario table (failure rate, avg duration, last error, click to open .feature), and last-run diagnostic card (same top-1 rule as the tree summary row).
• Evidence links on failures (screenshots, traces, videos when present).
• Actionable diagnostics: missing SDK from global.json, private NuGet feed/auth errors, vulnerability-as-error, filter mismatches, broken Playwright drivers, pending step definitions, etc. Surfaced in Output (summary or full), post-run toast, tree summary chip, and dashboard card.
• AI-ready failure context: copy structured markdown of the last failed run to the clipboard for Cursor/Copilot (no embedded LLM — review before sharing externally). Optional bddPilot.ai.rehydrateFromTrx (default off) rebuilds that context after Reload from the latest Pilot TRX; the markdown notes when the source was rehydrated (may be stale).
• Post-run feedback: error snippets on failed scenarios (hover + description), localized outcomes, optional summary toast (bddPilot.feedback.postRunToast).

Iconography

BDD Pilot uses VS Code codicons for actions and outcomes, plus two brand assets that share the same silhouette (radar + pilot). Pilot is a Gherkin cockpit alongside Test Explorer, not a TE replacement.

Cockpit signals (why this is not TE)

ForgeOne family: BDD Pilot = execution · BDD Guardian = navigation & step bindings · BDD Jarvis = workspace insights (consumes Pilot’s read-only Run API). Complementary extensions — not a monorepo and not an “AI test fixer.”

Security

• The extension never reads or stores credentials. Secrets continue to come from the project's own .env mechanism.
• An optional config/.env.<stage> file is loaded into the test process's environment in memory only (never logged or persisted).
• All output is sanitized before being written to the channel (passwords, tokens, JWTs, connection strings, AWS access key ids, PEM private keys, and similar patterns are redacted).
• Running against stg/prod requires an explicit modal confirmation (configurable via bddPilot.requireConfirmationForStages).
• Production opt-in: bddPilot.security.allowProductionRuns defaults to false — STAGE=prod is blocked until you enable the setting; with it on, the production confirmation modal still applies.

Optional config/.env.<stage> files

BDD Pilot can merge stage-specific variables into the test process when you run from VS Code. This is optional — your project may already load its own .env files inside hooks or step definitions.

1. Create a config/ folder next to (or above) your test .csproj.
2. Copy config/env.example to config/.env.test (or .env.dev, .env.stg, .env.prod).
3. Select the matching STAGE in the status bar execution hub before running.

Load order: config/.env.<stage> → config/.env.<stage>.local → config/.env.local (each step overrides earlier keys). Values are merged in memory only; see Security above. Gitignore per-stage locals such as config/.env.*.local and/or config/.env.local in your repo.

The samples/minimal-bdd project includes a tracked config/.env.test placeholder for dogfood and Capa B verification.

Architecture

src/
├── core/          # Pure logic, no VS Code API — unit tested
│   ├── gherkin/   # .feature parser, grouping, discovery
│   ├── runner/    # dotnet test arg/env building + spawn
│   ├── results/   # TRX + Cucumber parsers, evidence, run history
│   ├── diagnostics/ # error-output analyzer
│   └── config/    # stages, modes, profiles, project locator, .env loader
├── providers/     # Tree, TestController, CodeLens, dashboard, RunService
├── security/      # env guard policy + output sanitizer
└── extension.ts   # activation + commands wiring

The core/ layer has no dependency on the VS Code API, so it is fully unit testable and reusable (e.g. a future CLI).

Configuration

Tree items use label = name only; tags and run metadata appear in a short description (e.g. 6 tags, 2.3 s). Hover shows full tag lists and duration as 2.3 s (2341 ms).

Requirements

• VS Code 1.90+
• .NET SDK (any feeds your project needs must be reachable / authenticated on your machine — BDD Pilot does not manage credentials)

Development

npm install
npm run compile      # type-check
npm run lint
npm run test:unit    # core unit tests (node:test) + sample smoke
npm run build        # bundle with esbuild -> dist/extension.js
npm run package      # produce a .vsix
npm run dogfood      # automated pre-release smoke (lint, tests, VSIX, sample dotnet test)
npm run release:prep # maintainer gate: dogfood + version↔CHANGELOG + VSIX checklist (no publish)
npm run docs:sync-check  # fail if src/ changed without CHANGELOG.md

Agent / CLI / MCP

VSIX (recommended): Install BDD Pilot from a .vsix. On VS Code 1.101+ or Cursor with MCP support, the extension registers an MCP server (BDD Pilot) in agent mode — no manual mcp.json required. After a failed test run, the extension writes TestResults/bdd-pilot-last-failure.json so the pilot_failure_context tool can run with only projectDir.

Repo dev / CI (clone required):

npm run pilot -- discover samples/minimal-bdd [--enrich] [--test-target MinimalBdd.csproj]
npm run pilot -- build-filter samples/minimal-bdd --tag smoke
npm run pilot -- failure-context --project-dir samples/minimal-bdd --log path/to/run.log
npm run pilot:mcp    # MCP stdio server (Cursor / Claude Desktop)

For manual MCP config (repo clone), copy config/mcp.json.example to .cursor/mcp.json and set absolute paths for bddPilotRepo and workspaceFolder. Set BDD_PILOT_WORKSPACE_ROOT to your BDD project root so tool paths stay within the workspace.

Security: MCP tool output may include test failure data. Paths are restricted to the workspace root; logs/TRX are size-capped and sanitized — still review before sharing with external AI (same as Copy for AI in the extension).

Agent recipes (Copilot / Cursor)

Enable BDD Pilot in the Chat tools picker (agent mode). Paths below assume workspace root = your BDD project (e.g. samples/minimal-bdd).

1. Map the repo — pilot_discover_bdd with projectDir: ".". Add enrich: true only when you need outline rows from dotnet test --list-tests (slower).
2. Build a filter — pilot_build_filter with scope: "tag" and tag: "smoke" (or feature / scenario scope).
3. After a failure — run tests from the BDD Pilot panel, then pilot_failure_context with only projectDir: "." (uses TestResults/bdd-pilot-last-failure.json).
4. Reopen workspace — on activate, Pilot rehydrates TRX outcomes; narrative skip reasons (not_in_trx / canceled from the last scoped run) restore when the TRX matches that run; if the latest TRX has failures, the last-failure artifact is written so step 3 works without a new run.

Repo CLI equivalents:

npm run pilot -- discover . --enrich          # optional enrich
npm run pilot -- build-filter . --tag smoke
npm run pilot -- failure-context --project-dir .

Press F5 in VS Code to launch the Extension Development Host.

Sample BDD project

samples/minimal-bdd/ is a minimal Reqnroll + xUnit project used for CI smoke (dotnet test) and to validate feature discovery / filter mapping in unit tests. Open that folder as the workspace to dogfood BDD Pilot on a clean layout.

Roadmap

See ROADMAP.md. Current release is v1.16.0. Requires BDD Guardian v0.8.3+ for optional pre-run binding checks. Works alongside BDD Guardian.

Contributing

Contributions are welcome! See CONTRIBUTING.md. BDD Pilot is open source under the MIT License.