AgentWatch Test Guard
AgentWatch flags code that weakens your tests and quality checks in JavaScript
and TypeScript projects:
| Pattern |
Examples |
| Skipped tests |
it.skip, test.skip.each(...), describe.skip, xit, xdescribe |
| Focused tests (they silently skip every other test) |
it.only, describe.only, fit, fdescribe |
| Disabled type checks |
// @ts-ignore, // @ts-nocheck |
| Disabled lint rules |
/* eslint-disable */, // eslint-disable-next-line, // eslint-disable-line |
| Weak tests |
A test whose only check is expect(total).toBeDefined() |
| Tests with no assertion |
A test that calls code but checks nothing |
| Removed assertions |
expect(total).toBe(30) deleted from a test, or a whole test deleted |
| Weakened assertions |
toBe(30) → toBeDefined(), toHaveBeenCalledWith(...) → toHaveBeenCalled(), toThrow('msg') → toThrow() |
These matter more when working with coding agents: "make the tests pass" can be
done by skipping the test instead of fixing the code. AgentWatch makes that
visible the moment it happens.
AgentWatch reports patterns, not authorship. It never says who wrote the code.
How it shows findings
- Squiggles and the Problems panel. New findings are warnings; findings that
were already there are information markers.
- The AgentWatch sidebar (eye icon in the activity bar) lists findings by
file. Click one to jump to the code.
Every finding has an origin that says how much AgentWatch knows:
| Origin |
Meaning |
| existing |
Present when the project was opened, or in the version it is compared with. |
| detected later |
First seen after opening, with no commit of the file to compare with (e.g. a new file, or any change in a project with no commits). |
| confirmed new |
Not in the last commit. |
Git is optional
With Git, files are compared with the last commit (HEAD), so changes made
while VS Code was closed are caught too.
Without a commit to compare with, AgentWatch uses the pattern rules only: it
reports every skip, .only, @ts-ignore, eslint-disable, weak test and test with
no assertion in the code. Removed and weakened assertions need a commit, because deleted code leaves no
trace. AgentWatch deliberately doesn't treat the first code it sees as correct,
since that code may already be weak.
Commands
| Command |
What it does |
| AgentWatch: Scan Workspace |
Re-scans every JavaScript/TypeScript file. |
Settings
| Setting |
Default |
Description |
agentwatch.useGitBaseline |
true |
Compare with the last commit when the file is in a Git repository. |
agentwatch.disabledRules |
[] |
Rule IDs to hide: skipped-test, focused-test, ts-directive, eslint-disable, weak-test, no-assertion, removed-assertion, weakened-assertion. |
agentwatch.excludeFolders |
node_modules, dist, out, build, coverage, .git |
Folder names never scanned. |
Privacy
Everything runs locally inside VS Code. No network requests, no telemetry.
Code is parsed, never executed.
Limitations
- Test functions are recognised by name (
it, test, describe, …) without
type information.
- Not detected: Mocha's
this.skip(), Vitest's skipIf, node:test's
{ skip: true } option, ESLint's /* eslint rule: "off" */ configuration comments,
and Chai property assertions such as expect(x).to.be.ok.
@ts-expect-error is not reported: it fails once the error it expects is gone,
so it can't silently outlive its reason.- Committing a removed assertion makes that commit the new comparison point, so the warning goes away.
- Deleting a whole test file is not reported.
- A weak test is judged by its code alone, so a true/false variable with an unusual name (
assert(allTrue)) can be flagged by mistake.
- "No assertion" can't see checks hidden in hooks or in helpers with unrelated names, and some projects write such tests on purpose. If yours does, add
no-assertion to agentwatch.disabledRules.
- Files larger than 1 MB are skipped (usually generated or minified).
Development
npm install
npm test # compile and run the unit tests
Press F5 in VS Code to start an Extension Development Host with AgentWatch loaded.
| |
