TestPulse — Azure DevOps Extension

Professional test report generation from Azure DevOps Test Plans — PDF, HTML, Word, Excel, PowerPoint — with full traceability, a deterministic quality score and a GO/NO-GO release verdict.

📘 Full documentation: https://docs.atconseil.info

What is TestPulse?

TestPulse integrates into the Test Plans menu in Azure DevOps and turns your test plans into professional, shareable reports. It runs entirely client-side (no backend), works on Azure DevOps Services (cloud) and Azure DevOps Server 2022.1 (on-premises), and ships with a bilingual English / French interface and exports.

Beyond reporting, TestPulse adds a quality layer: a deterministic A–E score with a GO/NO-GO verdict, watch points, requirements coverage and drift analysis — to help teams decide whether a release is actually ready.

Key capabilities

Reporting & exports

• One-click test reports with KPIs and metrics from the plan(s) and suites you select.
• Five export formats: PDF, HTML, Word (.docx), Excel (.xlsx) and PowerPoint (.pptx) — and you choose which export buttons are visible.
• Run attachments (PNG/JPEG inline, other files listed), inline test-step detail, test-case tags, and ADO run links.
• Customizable PDF: theme color, cover logo, drag-and-drop section order, configurable signature page, and charts rendered locally (works offline).
• Robust cover page with dynamic layout (long titles and values never overlap), and a per-report executive summary (Automatic/Custom, light markdown formatting, optional forced verdict).

Traceability & quality

• Full Epic → User Story → Test Case → Bug traceability (Table or Cards view), per-US/Epic status aggregation, status filter, and automatic bug de-duplication.
• Quality score A–E + GO/NO-GO verdict from a single deterministic engine, with a transparent breakdown. An open blocking bug caps the grade at C and forces NO-GO.
• Audited decision override: overlay a human GO/NO-GO decision on the computed verdict without touching the engine — opt-in per project (off by default, disabled by the Strict profile), with a mandatory justification when it differs, captured author + date, and computed → decided shown together (web, widget, PDF, Word). A manual GO crossing a hard floor is flagged in red with the reason; the A–E grade stays computed.
• Open bugs on top: a concise block of open bugs at the top of the report (PDF/HTML/Word), sorted by severity then priority, shown whenever there is at least one open bug.
• Test coverage signal on the work item: a read-only “Test coverage” tab on a User Story / PBI (auto-placed, mounted on click) — an instant headline (Coverage + Depth pills) then an auto-loaded detail: depth findings, a numeric summary, and the linked tests’ latest pass/fail with the bug behind each failure and failures with no defect. A scope selector (Xray-style) re-frames the pass/fail on Latest (all plans) or a specific test plan — masking out-of-plan tests, remembered per work item, switching instantly in memory (no new read) — while Coverage/Depth/bugs stay plan-independent. A prominent execution-verdict banner (Xray-style) tops the detail — Passed / Not OK / Not Run / Not covered — a pure reduction of the scoped outcomes (Failed > Not Run > Passed; Blocked/NotApplicable = Not Run; no “partial”), with explicit colour + icon + label, re-computed in memory with the scope, and never recolouring the Coverage card. A story with no linked test shows a neutral state. Live and independent of published reports; reuses the depth/status/failure engines; no new scope or dependency, never writes, never reads comments. The scope selector stays visible whenever the story is covered — so results are always explicitly framed as the latest execution — and each linked test shows a clickable #ID that opens the work item in a new tab. The tests table is sortable (F-40): click any of the five headers (#, Test, Outcome, Last run, Bug) to sort, click again to reverse — remembered per work item, default order unchanged — and it rebounds in one click to the test run (executed rows) and to the scoped plan (new-tab portal navigations reusing the already-read run id; no new call/scope/dependency). When a test case covers several sibling stories that only differ by test configuration, the “Last run” now resolves from this story’s own configuration rather than a sibling’s more recent run, and both the run and plan links open the exact result / exact executed suite (F-44); the “Modified after last run” badge can be expanded (one lazy read, cached) to list what changed — the business fields, before → after, with the added wording highlighted and a “feeds depth analysis” badge — plus any attachment added or removed since the run (F-45).
• Watch points: surfaces what already weighs on the verdict — open blocking bugs, uncovered in-scope requirements, and unexecuted in-scope user stories — each as a clickable link.
• Scale profiles (project-level: Strict / Standard / Ramp-up) so everyone grades a report identically, plus an optional risk-based weighting confidence indicator.
• Failures without defect: flags failed test cases that have no linked bug, with one-click pre-filled native bug creation.
• Step-marking maturity metric and end-to-end Test Configurations support (per-config status, TC × configuration matrix, pass-rate weighting).

Exploratory testing

• Exploratory sessions on the work item: an “Explore” tab on a User Story / PBI runs a complete exploratory session from the story itself — a one-screen start (charter pre-filled with the story title, optional timer, free campaign tag, area/iteration inherited and editable through tree pickers), timestamped notes, and Ctrl+V to paste a capture of any screen or application (Win+Shift+S — desktop apps included, nothing to install). Captures open in an annotation editor (arrow, rectangle, text, highlighter, blur for sensitive data — destructive on save; two explicit colours, undo) and are attached to Azure DevOps only on validation; cancelling uploads nothing. Drag & drop attaches any file. From the session, create a bug with just a title and a severity — repro steps generated from the notes, captures, links and classification are injected, visible and editable (task and test-case variants included). Ending the session archives it as a Task tagged exploratory-session with the full report and every relation, making sessions natively queryable. Sessions autosave and restore automatically; a failed archive is retryable (a document-version conflict that could occasionally break closure on a restored session is fixed) and any write failure across the tab now shows a readable message instead of a raw error. A closed session can be exported as a signed PDF report — screen captures embedded as images, falling back to a plain name if one can't be fetched — either on its own or consolidated across every closed session of the story. Requires the vso.work_write permission (admin re-approval after update — a clear banner explains it until then).

Test Repository

• Repository tab: a free, project-shared folder tree to organise and find your Test Cases (Xray-style), storing references only — never a copy of a test case's content, title/steps always re-read live. Create/rename/delete/move folders at any depth via native drag & drop (confirmation before deleting a non-empty folder, anti-cycle guard); rank a case by dragging it onto a folder or via a fully keyboard-accessible "Move to…" menu (one folder max). Search by title/id/tag (accent- and case-insensitive), an always-recomputed Unclassified node, and a "Used in N location(s)" panel per case (plan › suite, no execution status). The Not in any plan node (formerly "Orphans") runs an explicit, on-demand scan (never automatic) for cases in no plan/suite, cached and re-runnable. Bootstrap the tree from Area Paths or the plan/suite hierarchy (numeric preview, purely additive, idempotent, zero new ADO endpoint). Bulk actions: multi-select bounded to the visible filter, assign/remove a batch in one atomic write with a summary. Readability: descendant-inclusive folder counts, a flat view with full path (view choice remembered), a non-blocking warning on same-named sibling folders. Rank: drag-and-drop or keyboard reordering, per-folder sort mode, explicit confirmation before discarding a manual order. Shared with optimistic concurrency — a single atomic Extension Data document (updated-by banner, no silent overwrite) — and load-time reconciliation that flags a case deleted in Azure DevOps as missing rather than dropping it. Generate a plan: materialise a folder into a real ADO plan/suites (new plan, existing plan, or existing suite), with a numeric preview before any write, deduplication, a snapshot-not-mirror guarantee, and a partial-failure report with no rollback — the Repository's only ADO write. Zero new scope, zero new dependency.

Coverage & drift

• Coverage builder: start from a set of requirements (area path / iteration / WIQL / IDs), see which are covered vs. gaps, preview a requirement-based regression plan, and export the spec (JSON / Markdown / CSV) — entirely read-only. Optionally create the plan in ADO (opt-in, confirmation required; create-only, never overwrites), choosing the target area path and iteration.
• Coverage depth (opt-in, off by default): a deterministic, AI-free qualitative assessment per requirement (Well covered / Partial / Shallow / Gap / Underspecified) with explainable findings and concrete improvement axes — advisory only, never blocking.
• Requirements drift: pick a baseline plan and a new one and see which requirements changed since the baseline run (a rendered-text diff names the field that moved), so you can target exactly what to re-verify — advisory and read-only.
• Stale coverage (F-43a): the Drift screen also lists requirements modified after the last run of their linked tests — a conditional “N Passed no longer prove anything” counter, the changed field (acceptance criteria / description / title, resolved lazily per requirement), filter chips and a project option to ignore title-only changes; the work item’s coverage tab shows a Modified after last run badge and the PDF/Word traceability a Stale column. Zero stale = a quiet “all fresh” line.
• Suite × Configuration matrix (F-43c): an internal report tab crossing every suite with every plan configuration (Passed / Failures / Partially run / Never run, including combinations with no point at all), a red count badge on the tab and a clickable Summary alert that only appear when gaps exist, team-shared “not applicable” markings excluded from the counts, and an opt-in grid in the PDF/Word exports (enable it in Settings > Report sections; disabled by default).

Trends, history & collaboration

• Execution cockpit (burndown): a per-plan daily campaign burndown in test points — remaining vs a piecewise ideal line rebased on the current remaining toward the plan end date, a projection of recent pace, and the suites/configs behind pace. Honest about a moving perimeter (add/remove/resize shown as labelled steps; a suite re-created for the same requirement reconciled as a re-org). Driven by small daily snapshots captured on any usage and stored like your saved reports — nothing leaves your tenant, no new API call, scope or dependency, read-only against test data. Set a local target when the plan has no end date; a closing nudge appears when a campaign stalls while its plan is still active. Readable & honest (F-41.2): dated axis ticks, a legend and point markers; the projection pauses after 14 days without execution (KPI “—”, never an invented date); a Progress panel always lists suites and configs (executed/total, independent of dates); and honest states — Campaign complete, execution dates unavailable on this instance, or a single-day note — replace any empty chart.
• What to run today & remaining effort (F-43b/d): under the Cockpit burndown, the plan’s remaining points ranked by a deterministic weighted score (last-run failure, requirement priority, open blocking bug, staleness) with three fixed project-level profiles (Strict / Standard / Onboarding), a per-row score breakdown and one-click execute links; plus a Remaining effort block — tester-days from the median run duration (P25–P75 range, per-suite medians with a flagged plan-median fallback), a weekend-aware finish date, two simulation sliders and the assumptions always displayed (an estimate, not a commitment) — echoed as one line in the report and its exports.
• Trends dashboard: pass rate, execution rate, bugs, flaky tests and historical requirements coverage over the last N reports per plan.
• Dashboard widget (read-only): pin "TestPulse — Quality overview" on any ADO dashboard — KPIs, trend and coverage from the latest project-published report, with no Test Plans access required (works for Stakeholders).
• Personal history with full-text search, CSV export and per-report notes; saved views; report sharing to the whole project; and a Draft → Submitted → Approved workflow.
• Import JUnit / NUnit / xUnit results (Cypress, Playwright, Surefire, NUnit, xUnit). Consolidated multi-plan reports with a per-plan breakdown.
• Teams / Slack notifications (webhooks, with a CORS copy-paste fallback for on-premises) and scheduled report reminders.

Configuration

• Configurable form fields: rename, hide, mark required, set a per-field soft length and regex validation, and choose which appear on the PDF cover. Plus configurable environment values, up to 3 custom fields, saved form templates, and a configurable PDF signature page.
• ADO user autocomplete for contact fields, and an in-app bilingual user guide with full-text search.

What's new in 2.40.0

• New visual identity (F-54): a teal signature colour, bundled Public Sans + JetBrains Mono typography (no external font loading — works fully offline on-prem), and consistent metric cards, badges, alert banners and buttons across all 8 tabs, replacing the old "default Azure DevOps" blue theme and its three inconsistent visual languages. The History table drops its plain black header for the same light header used everywhere else. Presentation only — every tab keeps its exact content, navigation and interactions; nothing to relearn. A dark theme is structurally prepared (two token sets) but not activated in this release.

What's new in 2.39.0

• Test Repository — generate a real ADO plan from a folder (F-53): the Repository's only ADO write. Three destinations — a new plan (Area/Iteration Path, optional reproduce the folder tree as static suites, depth preserved), an existing plan (cases added under its root suite), or an existing suite (its tree loaded lazily, only static suites selectable — requirement-based ones greyed out with an explanation). A numeric preview — suites to create, suites reused by name (never duplicated), cases to add after deduplication, cases already present and skipped — is shown before anything is written; nothing posts until "Create in ADO". The plan is a snapshot, not a mirror: editing the folder afterwards never touches it, and vice versa. Adding to an existing plan/suite carries a distinct warning (shared, possibly in-progress structure). No rollback on partial failure — the report lists exactly what was created and what failed, with a link to the plan.

What's new in 2.38.0

• Test Repository — organise a real project (F-52): seed the tree automatically from Area Paths or the plan/suite hierarchy — a numeric preview (folders to create/reused, cases to rank/already ranked) before any write, purely additive and idempotent, zero new ADO endpoint. Bulk actions: tick cases (selection survives scrolling, resets on folder/filter change), select-all bounded to the visible filter, assign/remove a batch in one atomic write with an exact summary. Readability: descendant-inclusive folder counts (Xray-style), a flat view listing every ranked case with its full path (view choice remembered), a non-blocking warning on same-named sibling folders. Rank: drag-and-drop or keyboard reordering (move up/down), per-folder sort mode (id/title/manual) with explicit confirmation before discarding a manual order. The Orphans node is renamed "Not in any plan" for clarity (same definition).

What's new in 2.37.0

• Test Repository reliability fixes: a folder read back without a stored parentId no longer empties the tree; re-running the Orphans scan or any subsequent tree edit no longer fails with a version conflict; and on Azure DevOps Services (cloud) the tab could occasionally 403 on its underlying work-item query — base URL resolution is now robust on both cloud and on-premises, with a genuine read failure now showing an explicit error banner instead of looking like an empty project. Under the hood, the tree now lives in a single atomic Extension Data document (replacing the original one-document-per-folder + index design), so a case move or a cascading folder delete can no longer leave a partial state — existing repositories are migrated automatically and non-destructively on next load.

What's new in 2.36.0

• Test Repository — a new "Repository" tab: organise and find your Test Cases in a free, project-shared folder tree (Xray-style), without ever touching Azure DevOps — it stores references only, never a copy of a test case's content. Create/rename/delete/move folders at any depth (native drag & drop, confirmation before deleting a folder that holds test cases); rank a case by dragging it onto a folder or via a fully keyboard-accessible "Move to…" menu — one folder max, "Remove from folder" sends it back to Unclassified (always recomputed, never stored). Search by title, id or tag, accent- and case-insensitive. Expand a case for "Used in N location(s)" (plan › suite, no execution status shown). The Orphans node runs an explicit, on-demand scan (never automatic) for cases in no plan or suite, cached and re-runnable — orphan status stays orthogonal to ranking. Shared with optimistic concurrency ("updated by X · refresh" banner, no silent overwrite) and load-time reconciliation that flags a case deleted in Azure DevOps as missing instead of dropping it silently. Zero ADO write, zero new permission scope, zero new dependency.

What's new in 2.35.0

• Exploratory sessions: reliable closure + PDF session report: ending a session could occasionally fail with an opaque [object Object] error on a restored session — the underlying document-version conflict is fixed, and every write failure across the tab (closure, abandon, capture upload, file drop) now shows a readable message. Abandoning a session that fails to save no longer silently pretends it worked — it stays active and retryable. New: Export PDF report turns a closed session into a shareable, signed PDF (charter, duration, full timeline, work items created) with screen captures embedded as images — falling back to a plain name if a capture can't be fetched, so the export never fails — and Export PDF report (US) consolidates every closed session of the story into one PDF. Both follow the active language (FR/EN).

What's new in 2.34.0

• Coverage signals get more precise (F-44/F-45): when the same test case covers several sibling stories that only differ by test configuration (Italy / Singapore / Korea…), the work-item “Test coverage” tab now resolves the “Last run” from this story’s own configuration, never a sibling’s more recent run — and the run/plan links open the exact result and exact executed suite instead of a blank page. The “Modified after last run” badge gains a “Show detail” button: expanding it (once, lazily) lists the business fields changed since that run — Acceptance criteria, Description, State, Priority, Iteration, Tags, Title — as before → after, with the added wording highlighted on rich fields via a home-made word diff (no library) and a “feeds depth analysis” badge, plus attachments added/removed. Read-only; the only new call is the lazy revision read, opt-in only.

What's new in 2.33.0

• Exploratory sessions — a new “Explore” tab on the work item: run an exploratory testing session from the user story itself — one-screen start (pre-filled charter, optional timer, campaign tag, inherited area/iteration), timestamped notes, Ctrl+V captures from any screen or desktop application annotated in place (arrow, rectangle, text, highlighter, destructive blur for sensitive data), drag-and-drop file attachments, two-field bug creation with generated repro steps and everything injected visible & editable (task / test case variants included), and session archiving as a queryable Task tagged exploratory-session carrying the report and all relations. Autosaved and restorable, bilingual, resilient to write failures. Adds the vso.work_write scope — an administrator must re-approve the extension after this update.

What's new in 2.31.0

• Suite × Configuration grid in exports is now opt-in: behavior change — the grid introduced in 2.30.0 no longer appears in PDF and Word exports by default. Re-enable it with the Suite × configuration grid checkbox in Settings > Report sections; the choice persists like every other section toggle. The grid layout, shared not applicable markings, report tab, badge and Summary alert are unchanged, and the grid still auto-hides when the plan declares no configuration.

What's new in 2.30.0

• Insights pack (F-43): four deterministic insights in their host screens — Stale coverage in Requirement Drift (requirements changed after their tests’ last run, lazy changed-field resolution, conditional counter, WIT badge, Stale column in exports), What to run today in the Cockpit (remaining points ranked by fixed weight profiles with per-row score breakdown and execute deep-links), a Suite × Configuration matrix in the report (gap-counting badge + clickable Summary alert only when gaps exist, shared not applicable markings, grid in PDF/Word), and Remaining effort in the Cockpit (median-based tester-days with range, weekend-aware finish date, simulation sliders, honest degraded state, one-line echo in the report and exports). Read-only; the only added call is the lazy revisions read for flagged requirements; no new scope or dependency.

What's new in 2.29.0

• Execution cockpit — daily campaign burndown: a new Cockpit hub tab turns TestPulse into a daily monitor — a per-plan execution burndown in test points (remaining vs a piecewise ideal line rebased on the current remaining toward the plan end date), a projection of your recent pace, and the suites/configs behind pace. It stays correct under a moving perimeter: add/remove/resize a suite or config shows as a labelled step, and a suite re-created for the same requirement is reconciled as a re-org. It runs on small daily snapshots captured on any usage (hub open or report generation) and stored like your saved reports — nothing leaves your tenant, no new API call, scope or dependency, read-only against your test data. No plan end date? Set a local target. A closing nudge appears when a campaign stalls while its plan is still active; setting the plan inactive freezes the burndown.

What's new in 2.28.0

• Coverage tab: sortable columns + open run / open plan (F-40): the “Test coverage” tab tests table is now sortable — click any of the five headers (#, Test, Outcome, Last run, Bug) to sort, click again to reverse (aria-sort); Outcome sorts failures-first via the same rank, Test by name (the [tag] prefix ignored), Last run chronologically (never-run rows last). The sort is remembered per work item and the default order is unchanged. Two one-click jumps are added: an executed row links to its test run (_testManagement/runs), and a scoped plan shows a Run in Azure DevOps link to the plan's execute view (_testPlans/execute) by the scope selector — both new-tab portal navigations (rel="noopener noreferrer"). The run id was already read, so there is no new API call, scope or dependency; presentation-only, read-only.
• Coverage plan insights & execute deep-links (F-42): in the Coverage builder, with Planning scan on, a Plans column shows per requirement where its linked tests are planned (plan name / N plans / —); expand a row to deep-link each (plan, suite) to the execute view and see a lazy n/m run progress (fetched only on expand, once per suite, degrades silently). The plan detail is parsed from the same suites?testCaseId responses the scan already receives — no extra API call at table load. Every application plan link now targets execute through one shared URL builder. Read-only; no new scope, dependency or endpoint.

What's new in 2.26.1

On-premises fix (F-35.4) — on Azure DevOps Server 2022.1, the "Test coverage" tab showed every test as Not Run even though they had passed. The plan/suite resolution call was sent with a project segment on a collection-scoped endpoint (404 on-prem; the cloud tolerated the extra segment). It is now collection-scoped with a legacy fallback and per-test-case resilience, so results read correctly. Read-only; no new API call, scope or dependency.

What's new in 2.26.0

• Execution-verdict banner on the work item (F-37): the “Test coverage” tab now leads with a colour-coded execution verdict — Passed (green), Not OK (red), Not Run (grey) or Not covered (neutral). It’s a pure reduction of the tests already shown, with strict precedence (any Failed ⇒ Not OK; otherwise any non-passed — Blocked, NotApplicable, NotExecuted… — ⇒ Not Run; only 100% Passed ⇒ Passed; no “partial”). It follows the scope selector (F-36) and re-computes in memory when you switch plans (no new read), shows icon + label (never colour alone, robust in light/dark), and never recolours the Coverage card — coverage (“is it tested?”) and execution (“does it pass?”) stay distinct axes. No new API call, scope or dependency.

What's new in 2.25.0

• Coverage scope selector on the work item (F-36): the “Test coverage” tab now lets you scope the pass/fail — Latest (all plans) (default, unchanged) or a specific test plan (Xray-style). Picking a plan masks the tests that aren't in it (with an “X of N in this plan” line), and re-scopes status, failures, failing-no-defect and freshness to that plan — while Coverage, Depth and the bug list stay plan-independent (a bug binds to a test case, not a plan). It switches instantly in memory with no new ADO call (the per-plan data is already fetched by the tab), is remembered per work item, and gracefully falls back to Latest if a remembered plan no longer applies. The selector only appears when a story spans 2+ plans. No new scope, dependency or write.

What's new in 2.24.4

• Test coverage as a tab on the work item (F-35.1): the coverage signal moves from a group in the Details tab to its own “Test coverage” tab on a User Story / PBI (auto-placed, mounted on click — so nothing is read until you open it). The tab opens with an instant headline (Coverage + Depth pills, no test-result reads) and then loads the detail automatically (no more manual expand): a depth-analysis block (F-30 findings), a numeric summary (failing / bugs / failing-no-defect + last executed), and a tests table with the linked tests’ latest pass/fail (failures first), the bug behind each failure, and the failures with no defect logged. A story with no linked test shows a neutral No linked test cases (no fetch); a read failure degrades to a quiet coverage unavailable. Still live and read-only, reuses the existing depth (F-30), traceability-status and failure-without-defect (F-20) engines, with a per-WI cache and no new permission or dependency. (The move to a page/tab contribution makes the extension register reliably in ADO.)

Getting started

1. In Azure DevOps, open your project → Test Plans → TestPulse.
2. Report tab: choose a plan (and suites), fill in the project information, then Generate report. Required fields are validated; non-blocking warnings appear in orange without stopping generation.
3. Review the report (Results, Traceability, Bugs, full history), then export using the format buttons at the top.

The EN / FR language selector is in the top-right corner.

Settings (overview)

Configure the report title and theme color, alert thresholds, export sections and visible export buttons, environment values, form fields (labels, visibility, required, validation, cover-page inclusion), custom fields, the PDF signature page, the project quality scale profile, and notifications / scheduled reminders.

Required permissions

Reporting, traceability, quality scoring, coverage analysis and drift are read-only. Write actions are explicit only: creating a test plan from the Coverage builder (vso.test_write), and — from an exploratory session — attaching captures, creating work items and archiving the session (vso.work_write). Adding a scope requires an administrator to re-approve the extension; until then the Explore tab shows a clear banner instead of raw errors. If you hit a "manage test plans under this area path" error, that is an Azure DevOps permission on the chosen area path — not a missing scope.

Troubleshooting

About

Publisher: Alexandre THIBAUD · LinkedIn: https://www.linkedin.com/in/alexandrethibaud/