MCP Doc Search

Semantic documentation search for any monorepo.

Large repos can have hundreds or thousands of markdown files of documentation. This extension helps developers manage them by enabling precise document retrieval—find and include only the relevant sections you need, dramatically reducing context bloat and token usage in AI assistant conversations.

• VS Code extension: type-ahead search in the command palette, auto-reindex on save, status bar indicator
• MCP server: search_docs, list_docs, reindex_docs, get, multi_get, plus per-file set_context / list_contexts / remove_context tools so any MCP-compatible AI assistant can find and read the right document in a single call
• Local embeddings: auto-downloads all-MiniLM-L6-v2 (ONNX, ~90 MB) on first use, then works fully offline — no API key required; multilingual-e5-small, EmbeddingGemma and nomic-embed-text-v1.5 are one setting away
• Heading-aware chunking: splits markdown on #/## boundaries, sizes chunks to the model's context window, never cuts a code fence or table, and prefixes every chunk with a [path › H1 › H2] breadcrumb
• Hybrid search: vector similarity fused with a BM25 full-text index (reciprocal rank fusion), so exact identifiers and German terms are matched literally, not only by meaning

Quick start

Install the VS Code extension

Install from the VS Code Marketplace:

code --install-extension de-otio.mcp-doc-search

Or grab a per-platform VSIX from the latest GitHub Release:

code --install-extension mcp-doc-search-<target>-<version>.vsix

Configure for your repo

Open VS Code settings and set:

Index location: Indexes are centralized under ~/.doc-search/indexes/ (outside your project tree). Any existing .doc-search-index folder is migrated there automatically on first run; if a global index already exists for the workspace, the now-redundant in-tree .doc-search-index is removed automatically on activation, so you never end up with both. The in-tree .doc-search-index layout (workspace mode) is deprecated — set docSearch.indexLocation to workspace only if you must keep it.

External roots: docSearch.extraRoots indexes directories outside the workspace alongside your docs — e.g. a locally cloned vendor-documentation repo:

"docSearch.extraRoots": [
  { "name": "vendor-docs", "path": "~/repos/vendor/docs" } // glob defaults to **/*.{md,mdx}
]

Their files appear in results as ext://vendor-docs/<path> and are fetchable through get/multi_get like any other ref. External roots are re-scanned on reindex (the save-time watcher covers only the workspace), and can be edited in the settings panel ("Doc Search: Open Settings" → "External folders") or directly in settings.json. Note that a configured root grants doc-search clients read access to that subtree — so the MCP server and CLI take it from the DOC_SEARCH_EXTRA_ROOTS environment variable only, never from a cloned repo's .vscode/settings.json (the generated .mcp.json carries your setting in its env block). Details and the opt-in in doc/configuration.md.

Use it

• Cmd+Shift+P → "Doc Search: Reindex Documentation" — build the initial index (takes ~30s for large repos)
• Cmd+Shift+P → "Doc Search: Search Documentation" — type-ahead semantic search, click a result to open it
• Cmd+Shift+P → "Doc Search: Generate .mcp.json" — creates .mcp.json so any MCP client can use the same index

Understanding scores

Each result includes a score (0–1): the cosine similarity between your query and the chunk's embedding.

Results are ordered by rank fusion, not by score alone (see Hybrid search): a chunk that matches your exact terms can appear above one with a higher similarity. A low-scoring hit near the top therefore usually means "found by the literal terms, not by meaning".

Pass explain: true to search_docs to get a detailed breakdown:

• vectorScore — cosine similarity from embeddings (same as score)
• vectorRank — position in the vector candidate list, or null if only the full-text side found it
• ftsRank — position in the full-text (BM25) candidate list, or null if no term matched literally
• rrfScore — the fused score the ordering is sorted by
• keywordTermsMatched — query terms found in the chunk text
• finalScore — same as score
• rank — position in result list (1-indexed)

Hybrid search

Every search runs two candidate lists per query and fuses them with reciprocal rank fusion (RRF, k = 60):

1. Vector — the query embedding's nearest chunks (top 3n, capped at 300)
2. Full-text — a BM25 inverted index over chunk text (top 3n). Terms are lowercased and matched literally; punctuation splits tokens (dot:workstream matches dot and workstream), there is no stemming, so identifiers, setting keys and German compounds hit exactly as written.

A chunk found by both sides rises to the top; a chunk the embedding misses but the words hit is still recovered. Phrase queries as one concept per call and include exact identifiers where you know them. To fuse several phrasings (a synonym, the German term, an identifier) in one round trip, pass queries: ["...", "..."] alongside query — at most 5 distinct queries in total.

The full-text index is built and refreshed by reindex. An index created before 0.8 has none until its next reindex; until then search silently ranks by vector similarity only (a warning is logged).

MCP integration

After running "Generate .mcp.json", connect any MCP-compatible client (Claude Code, Cursor, etc.). The generated config is portable — it points at the stable launcher as ${HOME}/.doc-search/bin/mcp-server.js (a forwarder the extension refreshes on every activation, so it survives upgrades) with DOC_SEARCH_WORKSPACE set to ${CLAUDE_PROJECT_DIR}, and carries your external roots and embedding provider in its env block, which is the only place the server reads them from. It is written with mode 0600 and gitignored. The MCP tools appear automatically:

search_docs("authentication flow")               → semantic search
search_docs("authentication", explain=true)      → same, with per-result score breakdown
list_docs()                                       → list every indexed file
get("doc/api.md")                                → read one file (full text)
multi_get("doc/**/auth*.md")                     → read many files in one call
reindex_docs(force=true)                         → full rebuild

# Per-file context notes the indexer carries alongside chunks
set_context("doc/api.md", "primary API reference")
list_contexts()
remove_context("doc/api.md")

If the client is an AI coding agent, see the Agent Guide for token-efficient usage patterns (chunk-level retrieval, multi-repo federation, delegating sweeps to subagents).

Embedding providers

The local provider runs one of four bundled-runtime models, chosen with docSearch.localModel (env DOC_SEARCH_LOCAL_MODEL for the MCP server and CLI): all-MiniLM-L6-v2 (default, English, ~90 MB), multilingual-e5-small (~95 languages incl. German, ~118 MB), embeddinggemma-300m (100+ languages, 768-dim, ~310 MB) and nomic-embed-text-v1.5 (English, 768-dim, 8k-token window, ~131 MB). Non-English docs should use one of the multilingual models; switching downloads the model once and rebuilds the index. See Configuration.

The OpenAI API key is stored in VS Code's SecretStorage (the OS keychain) — never in settings.json. For the standalone MCP server and CLI, export OPENAI_API_KEY in your shell; the generated .mcp.json (via Doc Search: Generate .mcp.json) references it as "${OPENAI_API_KEY}", which Claude Code expands at launch, and never contains the key itself.

Doc Search probes the provider before a large reindex and stops with a specific error — server unreachable, model not pulled, bad key — rather than failing file by file. One case is worth knowing about: upgrading Ollama does not restart the running server, and a stale daemon keeps answering /api/version while every model load fails, so indexing looks like it has hung. Doc Search offers to restart it for you; by hand it is brew services restart ollama (or systemctl --user restart ollama). See Ollama stops embedding after an upgrade.

CLI

A standalone CLI is included — no MCP client required. The extension keeps an upgrade-stable copy at ~/.doc-search/bin/mcp-doc-search.js (run it as node ~/.doc-search/bin/mcp-doc-search.js …).

# Semantic search
mcp-doc-search search "authentication flow" --n 5
mcp-doc-search search "map view feed" --files          # one path per line
mcp-doc-search search "query" --min-score 0.7 --json   # JSON output

# Browse the index
mcp-doc-search list
mcp-doc-search list --json

# Rebuild the index
mcp-doc-search reindex
mcp-doc-search reindex --force   # re-embed every file

# Read files from the workspace
mcp-doc-search get doc/api.md
mcp-doc-search get doc/api.md --from-line 20 --max-lines 50

# Read multiple files (glob or comma list)
mcp-doc-search multi-get "doc/**/*.md" --files         # list matched paths
mcp-doc-search multi-get "doc/a.md,doc/b.md" --json

# Index health (file counts plus the recorded embedding model and chunking settings)
mcp-doc-search status
mcp-doc-search status --json

# Per-file context notes carried alongside the index
mcp-doc-search context add doc/api.md "primary API reference"
mcp-doc-search context list
mcp-doc-search context remove doc/api.md

Flags: --json (machine-readable output), --files (paths only, for search/multi-get), --explain (score breakdown for search).

Environment: same as the MCP server — DOC_SEARCH_WORKSPACE, DOC_SEARCH_GLOB, DOC_SEARCH_EXTRA_ROOTS, DOC_SEARCH_HOME, DOC_SEARCH_INDEX_LOCATION, DOC_SEARCH_INDEX_DIR, USE_OPENAI=1, OLLAMA_URL, OLLAMA_MODEL. External roots and the provider are read from the environment only (see Trust model).

Exit codes: 0 = success, 1 = user error (bad args / missing file), 2 = engine error.

HTTP daemon mode

By default, each MCP client spawns the server as a short-lived stdio subprocess. The embed model takes ~1–2 s to load on cold start. Running a long-lived HTTP daemon amortises that cost across all clients.

Start the daemon

# One-shot foreground (useful for smoke-testing)
node dist/mcp-server.js --http --port 8181

# Detached daemon (parent exits, child runs in background)
node dist/mcp-server.js --http --port 8181 --daemon
# → MCP daemon started (PID: 12345, port: 8181)

# Verify it's up
curl http://localhost:8181/health
# → {"status":"ok","uptime":3.1}

Stop the daemon

node dist/mcp-server.js --stop
# → stopped (PID: 12345)

Point Claude Code at the HTTP endpoint

Edit your .mcp.json (or ~/.claude.json) to use the http transport:

{
  "mcpServers": {
    "doc-search": {
      "type": "http",
      "url": "http://localhost:8181/mcp"
    }
  }
}

vs stdio transport (the default, spawns a new process per client):

{
  "mcpServers": {
    "doc-search": {
      "type": "stdio",
      "command": "node",
      "args": ["/path/to/dist/mcp-server.js"],
      "env": { "DOC_SEARCH_WORKSPACE": "/path/to/your/repo" }
    }
  }
}

Idle model disposal

After 5 minutes of inactivity, the daemon automatically releases the embed pipeline from memory. The next request transparently reloads it (~1 s penalty), then stays fast again.

Loopback only, no browser access

The daemon binds 127.0.0.1 and answers only requests whose Host header is 127.0.0.1:<port> or localhost:<port>; anything else, and any request that carries an Origin header, gets 403 before it reaches the MCP transport. That closes DNS-rebinding and cross-origin calls from a web page on the same machine. CLI and IDE MCP clients (Claude Code, VS Code, curl) send neither header, so they are unaffected. There is no authentication beyond that: any local process running as you can reach the daemon, just as it can read the workspace directly (see SECURITY.md).

Development

npm install
npm run build       # bundle extension.js + mcp-server.js
npm test            # unit tests
npm run test:coverage  # coverage report
npm run package     # build .vsix for current platform

Platform-specific builds

LanceDB ships native binaries. Build for each platform:

npm run package:darwin-arm  # macOS Apple Silicon
npm run package:darwin-x64  # macOS Intel
npm run package:linux-x64   # Linux
npm run package:win-x64     # Windows

Architecture

src/
  core/          # Shared engine (no VS Code or MCP deps)
    types.ts     # DocChunk, SearchResult, EmbedProvider interfaces
    chunker.ts   # Markdown heading-aware chunking with fence detection
    embedder.ts  # LocalEmbedder, OllamaEmbedder, OpenAIEmbedder
    vectorstore.ts  # LanceDB wrapper (file-backed, cosine metric)
    searcher.ts  # Hybrid search: vector + full-text (BM25), reciprocal rank fusion
    indexer.ts   # Crawl, chunk, embed, upsert with mtime cache
  extension/     # VS Code extension shell
  mcp/           # MCP server: stdio + HTTP daemon transports
bin/             # Standalone CLI entry point

Three build outputs:

• dist/extension.js — VS Code extension host
• dist/mcp-server.js — standalone Node.js MCP server (stdio / HTTP daemon)
• dist/mcp-doc-search.js — standalone CLI binary

Contributing

Contributions are welcome — see CONTRIBUTING.md for setup, test, and PR conventions. By participating you agree to abide by the Code of Conduct.

Security

If you believe you've found a security issue, please follow the disclosure process in SECURITY.md. Do not open a public GitHub issue for suspected vulnerabilities.

License

MIT — see LICENSE.