The problem AI editors don't talk about
AI-generated code compiles, typechecks, and looks correct. It still ships bugs that pass review and break in production. We've audited thousands of agent diffs and these are the patterns that keep landing:
// 1. Phantom routes — handler never exists on the server
await fetch('/api/payments/confirm');
// 2. Swallowed errors — failures vanish silently
try {
await chargeCustomer(amount);
} catch (error) {
/* …nothing. ever. */
}
// 3. Hard-coded mock data shipped to prod
const revenue = 99_999;
const subscribers = 1_337;
// 4. Hallucinated dependencies
import { encrypt } from 'crypto-utils-pro'; // package doesn't exist on npm
// 5. Env-var ghosts
const key = process.env.STRIPE_PROD_KEY; // never declared, never set, always undefined
// 6. Drifted contracts — frontend payload doesn't match backend schema
await api.post('/users', { full_name: name }); // server expects { fullName }
Type-checkers, ESLint, Prettier, and the language server all sign off on every single one of those. VibeCheck doesn't.
How VibeCheck works
VibeCheck is a scan engine + verification layer that audits your repository the way a senior engineer would — by reading the code against itself, not against generic style rules.
┌─────────────────────────────────────────────────────────────────┐
│ Your editor │
│ VS Code / Cursor / Windsurf ◀──▶ VibeCheck Sidebar │
└─────────────────────────────────────────────────────────────────┘
│
│ triggers
▼
┌─────────────────────────────────────────────────────────────────┐
│ 16 engines → routes · deps · envs · auth · secrets · halluc │
│ contracts · types · tests · perf · … │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ Findings → trust score · evidence · fix plan · ship verdict │
└─────────────────────────────────────────────────────────────────┘
│
┌─────────────────┼──────────────────┐
▼ ▼ ▼
Auto-fix Ship Check Agent Firewall
(one-click patch) (release decision) (block bad AI edits)
The 6 work modes
The sidebar groups everything into six modes so you can move from finding to ship without context-switching:
| Mode |
Color |
What it's for |
| 🔍 Scan |
green |
Run a quick or deep scan. See findings stream in live. |
| 🧠 Understand |
cyan |
Architectural map of routes, deps, and call graphs. Spot ghosts visually. |
| 🛠 Fix |
amber |
Apply targeted fixes. Diff, run, verify. |
| 🚀 Ship |
red |
Release gate — get a score and a proof bundle before a deploy. |
| 🛡 Protect |
orange |
Agent Firewall — watch AI edits in real time and block risky ones. |
| 🎨 Studio |
violet |
Roast Me, prompt packs, mission generator, repo surgeon. |
Web companion: The same modes are available at vibecheckai.dev if you'd rather drive scans from a browser tab.
The 16 engines (one-line each)
| Engine |
Catches |
routes |
API endpoints called from the client but never implemented on the server |
deps |
Imports of packages not in package.json (or vice-versa) |
envs |
process.env.X reads where X isn't declared in any .env* |
auth |
Endpoints that look protected but aren't, or protected endpoints with broken middleware |
secrets |
Hard-coded API keys, tokens, JWT secrets |
halluc |
Calls to functions / classes / packages that don't exist |
contracts |
Request/response payload drift between client and server |
types |
Lossy as any / // @ts-expect-error casts that mask real bugs |
tests |
Functions claiming "fully tested" with no real test coverage |
perf |
N+1 queries, unbounded loops, blocking I/O in hot paths |
errors |
Empty catch blocks, swallowed promise rejections |
mocks |
Mock data, lorem-ipsum strings, demo IDs left in shipping code |
db |
Schema drift between migrations, ORM models, and live DB |
commit |
Commits that touch shipping code but don't touch related tests/docs |
doc |
README / docs that claim features the code doesn't implement |
policy |
Custom org-specific rules you bring via .vibecheck/rules/* |
Each engine emits findings with severity, file/line, evidence, and (where possible) a suggested fix.
Quickstart (90 seconds)
1. Install
Install VibeCheck — AI Code Safety Layer from the VS Code / Cursor / Windsurf Marketplace.
2. Scan
Open a project and press:
Cmd/Ctrl + Shift + V
You'll see live findings in the sidebar within a few seconds.
3. Triage
Click a finding. The right inspector shows:
- The code, with the exact lines highlighted
- The evidence trail (why we think it's broken)
- A proposed fix, runnable with one click
4. Ship
When you're ready to merge, run Ship Check (Cmd/Ctrl + Shift + P → VibeCheck: Ship). You get a single 0–100 score, a verdict (SHIP / BLOCK / WATCH), and a JSON proof bundle you can attach to the PR.
Keyboard shortcuts
| Shortcut |
Action |
Cmd/Ctrl + Shift + V |
Scan current file |
Cmd/Ctrl + Shift + A |
Quick actions menu |
Cmd/Ctrl + Shift + F |
Fix issue at cursor |
All bindings are rebindable via VS Code keyboard preferences.
Most-used commands
Open the command palette (Cmd/Ctrl + Shift + P) and type VibeCheck:.
| Command |
Mode |
Purpose |
VibeCheck: Scan |
Scan |
Full or quick repo scan |
VibeCheck: Scan Current File |
Scan |
Single-file scan |
VibeCheck: Open Scan Hub |
Scan |
Pick scan type (full / deep / api / security / deps / docs / polish / context) |
VibeCheck: Explain Current File |
Understand |
Plain-English rundown of what a file does + risk hotspots |
VibeCheck: Open Context Engine |
Understand |
Project map, route graph, dependency cluster view |
VibeCheck: Fix Selected Finding |
Fix |
One-click patch for the highlighted finding |
VibeCheck: Generate Repair Mission |
Fix |
Multi-step fix plan you can hand to an agent |
VibeCheck: Ship |
Ship |
Release-gate verdict + proof bundle |
VibeCheck: Doctor |
Ship |
Health check — env, DB, auth, deps, build readiness |
VibeCheck: Protect Repo |
Protect |
Toggle Agent Firewall on the current workspace |
VibeCheck: Open Reality Mode |
Protect |
Run real browser tests on your dev preview |
VibeCheck: Roast Me |
Studio |
Brutal critique of your repo (professional / spicy / brutal tones) |
VibeCheck: Open Repo Surgeon |
Studio |
Refactor + decomposition planner |
VibeCheck: Open Code Split |
Studio |
Surgical large-file split with safety checks |
VibeCheck: Open Command Center |
Any |
Unified hub for every mode |
There are 60+ commands in total. The palette filters them as you type.
Configuration
Open VS Code settings (Cmd/Ctrl + ,) and search vibecheck to see all options. The ones that matter most day-to-day:
| Setting |
Default |
What it does |
vibecheck.scanOnSave |
false |
Re-scan the file on every save |
vibecheck.scanOnOpen |
false |
Scan a file the first time it's opened |
vibecheck.realtimeScan.enabled |
true |
Live in-editor scanning while you type |
vibecheck.realtimeScan.debounceMs |
400 |
Debounce window for live scans |
vibecheck.watchMode |
false |
Background re-scan on file change events |
vibecheck.engines |
["*"] |
Allowlist / denylist of engines (e.g. ["routes","deps","-perf"]) |
vibecheck.ignorePaths |
[] |
Globs to skip — adds to .vibecheckignore |
vibecheck.severity.error |
true |
Surface error-severity findings as Problems |
vibecheck.severity.warning |
true |
Surface warnings |
vibecheck.realityReveal |
true |
Enable Reality Mode browser-test integration |
vibecheck.cliPath |
"" |
Override CLI binary path (falls back to bundled) |
Sign in (Device Flow)
When you trigger a feature that needs your account (Ship, cloud history, MCP), the extension opens a browser to:
https://vibecheckai.dev/approve?user_code=ABCD-1234
You log in once, approve the device, and the token lands in VS Code SecretStorage — never written to disk in plaintext, never exposed to other extensions.
CLI
Same engines. Same findings. Different surface.
# scan + render to terminal
npx @vibecheck-ai/cli scan .
# ship verdict for CI
npx @vibecheck-ai/cli ship --ci --format sarif > vibecheck.sarif
# guard mode — exit non-zero on critical findings
npx @vibecheck-ai/cli guard . --max-critical=0
CI integration
Drop this into .github/workflows/vibecheck.yml:
name: VibeCheck
on: [pull_request]
jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npx @vibecheck-ai/cli ship --ci --format sarif > vibecheck.sarif
- uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: vibecheck.sarif
SARIF output renders inline in the PR Files Changed tab — every finding appears next to the line that triggered it.
MCP server
VibeCheck exposes its tools to any MCP-capable agent (Claude Desktop, Cursor's agent panel, custom orchestrators).
npx @vibecheck-ai/mcp
Add to .mcp.json:
{
"mcpServers": {
"vibecheck": {
"command": "npx",
"args": ["@vibecheck-ai/mcp"]
}
}
}
Available tools: vibecheck_scan, vibecheck_protect, vibecheck_findings, vibecheck_fix, vibecheck_roast.
Plans
| Plan |
Best for |
Includes |
| Free |
Quick local confidence |
Unlimited local scans, issue counts, severity, trust score, 16 engines, sidebar |
| Pro |
Production handoff |
Full evidence, line-level detail, fix suggestions, scan history, SARIF, CLI, MCP, Ship verdict, project-wide scans |
| Team |
Multi-repo orgs |
Pro + shared workspace, role-based access, central policy, audit log |
| Enterprise |
Compliance & scale |
Team + SSO/SAML, custom engines, on-prem connectors, SLAs |
See vibecheckai.dev/pricing for current pricing.
Privacy & security
| Principle |
Stance |
| Local by default |
Scans run in-process. Your source never leaves the machine unless you opt into a cloud feature. |
| Token storage |
Auth tokens use VS Code SecretStorage (Keychain on macOS, Credential Manager on Windows, libsecret on Linux). |
| Trusted workspaces |
In Restricted Mode, scanning is read-only. Auto-fix, AI providers, telemetry, and shell-out commands stay disabled until you trust the workspace. |
| Virtual workspaces |
Static analysis works in virtual filesystems (remote repos, vscode.dev). Reality Mode, git hooks, and the ISL Studio MCP require local FS. |
| Telemetry |
Off by default. If enabled, only aggregate counters — no source code, file paths, or finding contents. |
| CSP / sandbox |
Webviews run with a strict CSP — no remote script execution, no inline event handlers. |
Compatibility
- VS Code ≥ 1.93
- Cursor (latest)
- Windsurf (latest)
- Node.js ≥ 20.11 (only needed if you use the bundled CLI from a custom path)
Works on macOS, Linux, and Windows. Apple Silicon and Intel both supported natively.
Troubleshooting
| Symptom |
Fix |
| "Authentication required" toast on activation |
Re-run VibeCheck: Sign In. Token may have expired. |
| Sidebar blank, dashboard webview won't load |
Reload the window (Cmd/Ctrl + Shift + P → Developer: Reload Window). If still blank, run pnpm --filter @vibecheck/dashboard-legacy build:webview from the repo root. |
| MCP tools missing in Claude Desktop |
Check npx @vibecheck-ai/mcp runs cleanly in a terminal. The MCP client only sees tools after a successful tools/list. |
| CLI not found from a custom path |
Set vibecheck.cliPath in settings to the absolute path of the bin. |
| Scan finds nothing |
Likely a .vibecheckignore rule. Run VibeCheck: Open Scan Hub → check the "Files scanned" count. |
If you hit something not in this table, open an issue at mailto:founder@vibecheckai.dev with the output of VibeCheck: Open Report.
Roadmap (next 90 days)
- Real-time Agent Firewall for Cursor's agent mode
- Inline AI explanations for every finding (opt-in, local model first)
- Custom engine SDK (
@vibecheck/engine-sdk)
- Team policy packs (org-wide rule bundles)
- Notion + Linear integration for missions