Post-Agent Checks

After the Copilot coding agent edits your code, this extension:

1. Finds what changed (via git status)
2. Lints each changed file with your configured linter
3. Runs a SAST scan (default: Semgrep) on each changed file
4. Generates or updates a test file for each changed source file, using Copilot's language model directly

Results go to the "Post-Agent Checks" Output panel, plus a summary popup with counts.

How it decides "what changed" (works with or without git)

1. Files saved during the current VS Code session (primary, most precise) — tracked in-memory as you/the agent save files. No git needed at all.
2. git status (secondary) — only used if #1 found nothing yet (e.g. right after a fresh reload, before any saves have happened) and the folder happens to be a git repo. Skipped silently if it isn't.
3. Recently modified files (last resort) — if neither of the above applies, scans the workspace for any file modified in the last postAgentChecks.mtimeScanWindowMinutes (default 15).

The Output panel logs which source was used for each run, e.g. Change source: session saves.

Why there's no fully automatic "agent finished" trigger

VS Code does not currently expose a public API event for "the Copilot agent just finished." (There's an open feature request for exactly this — microsoft/vscode#310951 — not yet implemented as of writing.) So this extension gives you two trigger options — use one or both:

Option A — Agent Hook (reliable, recommended)

VS Code agents support hooks: scripts that run at specific points in an agent session, including when a session ends. Wire the sessionEnd hook to run this extension's command via the VS Code CLI.

Create/edit your hooks config (workspace .vscode/hooks.json, or your user hooks file — check VS Code's current Hooks documentation for the exact file location, as this is an evolving feature) with something like:

{
  "hooks": {
    "sessionEnd": [
      {
        "command": "code --command postAgentChecks.runChecks"
      }
    ]
  }
}

This fires the moment an agent session ends — no guessing, no delay.

Option B — Idle heuristic (works out of the box, less precise)

Toggle "Post-Agent Checks: Auto" in the status bar (bottom left), or set in settings:

"postAgentChecks.autoRunOnIdle": true,
"postAgentChecks.idleSeconds": 8

The extension watches file saves; if edits go quiet for idleSeconds, it assumes the agent just finished a batch of changes and runs checks automatically. This can false-trigger on your own manual edits too — it's a reasonable default when you don't want to touch hook config, but Option A is more precise.

Option C — Manual

Command Palette → "Post-Agent Checks: Run Now" — run it whenever you want, e.g. right after watching the agent wrap up.

Setup

npm install

You'll also want the actual tools installed/available in your project, matching your config:

• Lint: whatever's configured per extension (defaults assume eslint for JS/TS, ruff for Python — via npx, so no global install needed if they're project devDependencies)
• SAST: Semgrep (default command is npx semgrep --config=auto --quiet) — install with pip install semgrep or use another SAST CLI by changing the setting
• Test generation: needs GitHub Copilot installed and signed in in the Extension Development Host window (it calls Copilot's model directly via vscode.lm)

Configure

All settings are under postAgentChecks.* — search for "Post-Agent Checks" in Settings, or edit .vscode/settings.json directly. Key ones:

{
  "postAgentChecks.enableLint": true,
  "postAgentChecks.enableSast": true,
  "postAgentChecks.enableTestGeneration": true,
  "postAgentChecks.lintCommands": {
    ".ts": "npx eslint",
    ".py": "npx ruff check"
  },
  "postAgentChecks.sastCommand": "npx semgrep --config=auto --quiet",
  "postAgentChecks.testFileNaming": {
    ".ts": "{name}.test.ts",
    ".py": "test_{name}.py"
  },
  "postAgentChecks.excludeGlobs": [
    "**/node_modules/**",
    "**/*.test.*"
  ]
}

Swap in whatever linter/SAST tool your project actually uses — these are just sensible JS/TS/Python defaults.

Run locally

1. code .
2. Press F5 — opens an Extension Development Host window
3. Open a git repo folder with some uncommitted changes
4. Command Palette → "Post-Agent Checks: Run Now"
5. Watch the "Post-Agent Checks" Output panel populate, then check the summary popup

Notes / limitations

• Test generation calls Copilot's model per changed file — for large batches of changes this can take a bit and will consume your Copilot usage, same as any chat request.
• Generated tests are a starting point, not a guarantee of coverage — always review them.
• The idle-heuristic trigger (Option B) can't distinguish "the agent finished" from "you just saved a file yourself" — prefer the hook (Option A) if that matters for your workflow.
• SAST/lint findings are written to the Output panel as raw tool output, not parsed into VS Code's Problems panel — say the word if you'd like that added (it's a reasonable follow-up: parse each tool's output format and push diagnostics via vscode.languages.createDiagnosticCollection).

Package as .vsix (optional)

npm install -g @vscode/vsce
vsce package
code --install-extension post-agent-checks-0.0.1.vsix