Docgrity for VS Code

Find where your repository's docs disagree with themselves — and raise a GitHub issue to get it fixed.

Install: search “Docgrity” in the Extensions view, or code --install-extension ujjavala.docgrity

The repo-docs sibling of the Docgrity Confluence app. Scoped deliberately: markdown files only (**/*.md — READMEs, ADRs, runbooks, guides).

This repo contains all three repo-docs surfaces:

See action/README.md for Action and CLI usage (uses: ujjavala/docgrity-vscode/action@main).

What it does

1. Scan — Docgrity: Scan repository docs collects your markdown files, picks candidate pairs locally with TF-IDF (no network), then asks the LLM to assess:
   • Contradictions — conflicting factual claims across two docs
   • Duplicates — substantially overlapping docs that should be merged
   • Open questions — unresolved TBD/TODO/"who owns this?" buried in docs
2. Review — findings appear in the Docgrity view with evidence excerpts; each excerpt is a click away from the exact spot in the file, and shows as a diagnostic squiggle. Every finding records the model + prompt version.
3. Act — right-click a finding → Raise GitHub issue. Docgrity drafts the issue (title, evidence, suggested next step, potential owner from git history), shows you the draft, and only creates it after you approve. The issue is labelled docgrity and docgrity:<type>.

Zero cost, zero keys

• All LLM calls go through your own GitHub Copilot subscription via the VS Code Language Model API. No API keys, no servers, no telemetry.
• Issue creation uses VS Code's built-in GitHub sign-in.
• Candidate selection is local TF-IDF — the LLM only sees the top pairs.

Modes: report-only vs report + issues

Set it in Settings → search “docgrity mode”, or in .vscode/settings.json:

{ "docgrity.mode": "report-only" }

Per-workspace settings win over user settings, so you can default to report-only globally and enable issues only in repos you own.

Choosing which checks run

Each check is a separate toggle — run any combination:

These combine freely with any model (docgrity.model.*) and either mode (docgrity.mode). Disabling checks also speeds up scans: pair selection is skipped entirely when both pairwise checks are off.

Scan performance

• When both pairwise checks are enabled they run as a single combined LLM call per pair (the model reads each pair once, not twice).
• Docs with no open-question signals (no TODO/TBD/???/unanswered questions) are pre-filtered out before any LLM call.
• Assessments run with bounded concurrency (4 at a time).
• For large repos, tune docgrity.maxFiles, docgrity.maxPairs, and docgrity.include to narrow the corpus.

Choosing your model (Copilot, Claude, GPT, local llama…)

Run Docgrity: Select AI model from the command palette — it lists every model VS Code exposes and saves your choice. Or set it manually:

Options, in order of simplicity:

1. Copilot (default) — sign in to GitHub Copilot; nothing to configure. Copilot models come via the vscode.lm API and are unaffected by GitHub's retirement of the separate GitHub Models service.
2. Claude / GPT / Gemini via Copilot — any model enabled in Copilot's model picker is available; set docgrity.model.family (e.g. claude-sonnet-4.5) or use Select AI model.
3. Local Ollama — install Ollama, ollama pull llama3.1, then in Copilot Chat → Manage models → add the Ollama model. It registers under the copilot vendor; pick it with Select AI model. Fully local — no doc content leaves your machine.
4. Remote Ollama over a Cloudflare Tunnel — if your model runs on another box (home server, GPU rig):

   # on the machine running Ollama
   cloudflared tunnel --url http://localhost:11434
   

   Point Copilot's Manage models → Ollama endpoint at the generated https://….trycloudflare.com URL. Note: quick tunnels get a new URL on every restart — re-update the endpoint each time, or create a named tunnel with your own domain for a stable URL (cloudflared tunnel create …). Protect a named tunnel with Cloudflare Access — an open LLM endpoint is abusable.

Small local models fail Docgrity's strict-JSON validation more often than hosted ones; failed responses are rejected safely (never mis-recorded) — expect fewer findings rather than wrong ones. 8B+ instruct models work best.

Outside the editor (the CLI and GitHub Action) Copilot is not available — there is no Copilot API for terminals or CI. Use ollama (keyless) or bring your own anthropic / openai / gemini key instead.

No-agent mode (no AI at all)

Set docgrity.engine to no-agent (CLI/Action: provider: none) to scan with pure algorithms — no model, no Copilot, no keys, fully offline:

• Duplicates — verbatim shared-block detection + TF-IDF similarity (catches copy-paste duplication; paraphrased duplication needs AI).
• Open questions — explicit markers (TODO, TBD, FIXME, ???, "open question"…) with deterministic high confidence.
• Contradictions — ❌ require AI intelligence (semantic understanding of conflicting claims). They are skipped in no-agent mode — you're told, never given heuristic guesses.

Evidence stays verbatim, owners still come from git history, and raised issues use a deterministic template. Confidence is shown on every surface — Problems view squiggles, the findings tree, reports, CLI output, and issue bodies — so you can always judge how sure a finding is.

Design principles (shared with the Forge app)

• Typed JSON outputs only — model responses are validated in code, never trusted prose.
• Every finding requires verbatim evidence, verified against the source file (hallucinated quotes are dropped).
• Ownership is always potential (last git author), never asserted.
• Nothing is posted anywhere without explicit human approval.
• Doc content is untrusted input — it cannot override agent instructions.

Requirements

• VS Code 1.95+, an active GitHub Copilot subscription, a workspace with a GitHub origin remote (for issue creation).

Development

npm install
npm run compile
# F5 in VS Code to launch the Extension Development Host

Settings