Hunt down flaky tests: import CI JUnit reports, track pass/fail history over time, score instability, detect root causes (timing, order, network, shared data) and suggest fixes.
Installation
Launch VS Code Quick Open (Ctrl+P), paste the following command, and press enter.
Hunt down flaky tests before they hunt your CI. Test runners execute tests — nobody analyzes which tests are unstable, why, and how to fix them. FlakyTest Hunter closes that gap.
What it actually does
Feature
Description
CI history ingestion
Parses JUnit XML reports (GitHub Actions, GitLab, Jenkins, pytest, surefire, jest-junit) and jest/vitest JSON results. Scan the workspace or import a folder of CI artifacts.
Flakiness detection
Rolling per-test history with dedupe; a test is flagged when it both passes and fails enough — scored 0–100 via balanced pass/fail ratio weighted by sample count.
Root-cause classification
Analyzes failure messages and duration variance to attribute flakiness: timing (timeouts, races, CV > 0.8), network (ECONNREFUSED, 5xx), shared data (duplicate key, locks), order (state leaks) — each with confidence and evidence.
The repo includes samples/junit-run-1..4.xml — four simulated CI runs containing two genuinely flaky tests. Open the folder in VS Code, run Scan Workspace for Test Reports, and open the dashboard.
Build the VSIX
npm install # no lockfile committed; npm ci would fail
npm run compile
npm run package # produces flakytest-hunter.vsix
Publishing via GitHub Actions
Every push builds and uploads the VSIX as an artifact. Push a v* tag to auto-create a GitHub Release and publish to the Visual Studio Marketplace (requires a VSCE_PAT secret).