Bespoke AI

AI autocomplete for prose and code in VS Code — powered by your Claude subscription or API key

Actively developed. Completion quality varies by model and context — see How quality is measured and Known issues. Open an issue if something looks wrong.

💻 macOS, Linux, and Windows — Also works in VSCodium (Open VSX).

🖊️ Writing, not just code — Inline completions for prompts, journals, notes, docs, and code. Matches your voice and style in prose; language-aware in code.

🔑 Subscription or API key — Use your Claude subscription through Claude Code (no per-token costs), or bring your own API key from xAI, OpenAI, Anthropic, Google, OpenRouter, or Ollama.

✨ One-click commit messages — Hit the sparkle button in Source Control to generate a commit message from your staged diffs.

✏️ Suggest edits — One-command typo, grammar, and bug fixes for visible text.

🔧 Context menu — Right-click to Explain, Fix, or Do custom actions on selected text. Opens an agent session in a terminal — Claude Code by default, or opencode (works with local models, any backend).

Screenshots

Writing — autocompleting a prompt to Claude:

Code — filling in a function body:

Commit messages — the sparkle button in Source Control:

🚀 Getting Started

1. Install from the VS Code Marketplace, or in VSCodium from Open VSX (search for "Bespoke AI" in the Extensions view either way)
2. On first launch, the extension detects whether you have the Claude Code CLI — if not, it offers to help you set it up or switch to API key mode
3. Start typing — completions appear as gray suggestion text after a ~2-second pause plus the model's response time (typically another 2–3 seconds on the Claude Code backend)

That's it. For API mode, set bespokeAI.backend to api and choose a model from the status bar menu. Details for each path below.

Triggering Completions

Completions appear automatically after a ~2-second pause (the relaxed preset). Press Alt+Enter to trigger one instantly at any time.

If Alt+Enter doesn't work, another keybinding is likely intercepting it. Open Keyboard Shortcuts (Ctrl/Cmd+K Ctrl/Cmd+S), search for alt+enter, and rebind any conflicts — Inline Chat is the most common one.

Change trigger behavior via the status bar menu: relaxed (~2s), eager (~800ms), or on-demand (Alt+Enter only).

Modes

Override via settings or the status bar menu.

💡 Why This Exists

I tried every open-source AI autocomplete extension I could find. Most handled code fine but fell apart with prose — breaking paragraphs mid-thought, injecting code syntax into journal entries, producing gibberish outside of source files. Nothing came close to Copilot for non-code text.

So I built my own. And since I was already paying for a Claude subscription, I realized I could wire it up to use Claude Code instead of raw API calls — getting frontier model completions (Haiku, Sonnet, even Opus) at no additional per-request cost. Built on the Claude Agent SDK, it took extensive prompt engineering.

The Claude Code backend uses your existing paid Claude plan (Pro, Max, Team, or Enterprise). Heavy use may be subject to Anthropic's rate limits. The API backend uses standard per-token pricing from your chosen provider.

🧪 How quality is measured

Autocomplete fails quietly — you just dismiss bad ghost text — so quality is measured offline against ~105 scenarios modelled on real use: journal entries, prompts to Claude, mid-document edits, gaps between paragraphs, and code in several languages. Each run generates completions through the production prompt and extraction code, runs seven deterministic checks, and has an LLM judge score what the user would actually see. Details and all numbers: evals/.

What this has caught so far:

• Reading outputs beats trusting the judge. An error analysis of the last full run (by an AI agent: every failed output, a spot check of passes, and a mechanical whitespace check over all 99) found the judge had passed 15 completions that glued two words together (behind + inadequate → behindinadequate) and had given opposite verdicts to identical completions. Those failures are now caught by deterministic checks, and the judge is being validated against human labels before its pass rates are treated as more than relative.
• A prompt fix for the missing space. Glued words at a word boundary: CLI Sonnet 59% → 0%, xAI Grok 90% → 0% (targeted test, 2–3 samples per scenario); in the full suite, missing-space failures went from 27 to 0 and 36 to 1. On the scenarios the fix does not target, the judge's pass rate was mixed: Sonnet slightly down, Grok up. GPT-4.1 Nano did not respond to any of five prompt variants.
• A whitespace bug on the Anthropic API path that doubled spaces and blank lines at the cursor (blank lines: 100% → 0% after the fix).

Latency on the Claude Code backend, from real use: p50 1.9–2.6 s, p95 4.1–6.1 s depending on model, measured from request to response and separate from the deliberate ~2 s pause before a request is sent.

CI replays 52 recorded model outputs through the current extraction and cleanup code on every push, so a change to that code shows up as a test diff without calling a model.

Known issues

• GPT-4.1 Nano often glues the completion to the previous word (no leading space). Prompting did not fix it; prefer another model for prose.
• Models sometimes copy the text after the cursor. The copy is trimmed, which can leave no suggestion at all. xAI Grok does this more often since the whitespace fix.
• The anthropic-haiku API preset sometimes returns nothing, and occasionally shows its own reasoning. In one test (348 requests per arm), 16–17% of requests produced no suggestion, mostly because the model closed its answer immediately or ran out of room while second-guessing itself. Rarely, text the model writes before its answer (such as a repeated line of the instruction) reaches the ghost text. Fourteen prompt variants did not fix this without breaking other completions (details); the Claude Code backend and the other presets are not affected in the same way.
• Cost is only shown for the Claude Code backend, where the CLI reports it; the API backend shows tokens only.

🧩 Available Models

The API backend includes presets for popular providers. Change the active preset via the status bar menu or the bespokeAI.api.preset setting.

API keys: Store keys via the Enter API Key command (Command Palette → "Enter API Key"). Keys are saved in your OS keychain. As a fallback, the extension also checks environment variables and ~/.creds/api-keys.env:

xAI → XAI_API_KEY · Anthropic → ANTHROPIC_API_KEY · OpenAI → OPENAI_API_KEY · Google → GEMINI_API_KEY · OpenRouter → OPENROUTER_API_KEY · Ollama → no key needed

Custom models: Run Bespoke AI: Add Custom Model from the Command Palette for a guided setup wizard. Any OpenAI-compatible endpoint works (LM Studio, Together, Mistral, etc.), plus Anthropic, Google, and OpenRouter. You can also add presets manually:

"bespokeAI.api.customPresets": [
  { "name": "My Llama", "provider": "openai-compat", "modelId": "llama3.2", "baseUrl": "http://localhost:1234/v1" }
]

⚙️ Configuration

All settings live under bespokeAI.* in VS Code settings.

📋 Commands

User types → Mode detection → Context extraction → LRU cache check
  → Debounce → Backend Router → Claude Code CLI or API → Cleanup → Ghost text

🔎 Seeing what the model saw

Every completion request produces a trace record: what the model was sent, what it returned, how long it took, tokens used, cost (when the backend reports it), and what happened to the result — shown, cached, empty, aborted, or error.

Run Bespoke AI: Show Recent Completions (or pick Recent Completions from the status bar menu) to browse the last 200 requests in the current window, newest first. Click a row to see the system instructions, the exact message sent, the raw model output, the ghost text that was shown, and a timing breakdown (debounce, pool-slot wait, model time). Commit-message and Suggest Edits requests appear in the same list.

Rendered from the view's own HTML with synthetic requests and VS Code's Dark Modern theme colours.

Records use the attribute names of the OpenTelemetry GenAI semantic conventions (gen_ai.request.model, gen_ai.usage.input_tokens, …; the conventions are still in development; the extension follows semantic-conventions-genai 1.42.0-dev, checked 2026-09-27). That makes the same records usable outside the editor:

• bespokeAI.trace.file appends them to ~/.bespokeai/traces.jsonl, one span per line (rotated at 5 MB; archives older than 7 days are deleted).
• bespokeAI.trace.otlp.endpoint exports them over OTLP/HTTP (JSON) to your own backend. See Privacy and data flow for what is sent and when.
• bespokeAI.trace.captureContent: false keeps prompt and output text out of trace records (the Recent Completions view, the trace file, and any export); timing, tokens, and outcomes are still recorded. It does not govern the Output channel log: at bespokeAI.logLevel: trace the extension writes full prompts there, and VS Code keeps output logs on disk. Leave logLevel at info or debug if that matters to you. logLevel can only be set in user settings, so a repository cannot turn it on.

🔍 Troubleshooting

Completions not appearing?

• Check the status bar — is it showing "AI Off"? Click to re-enable.
• If it shows "Setup needed", the Claude Code CLI may not be installed or authenticated.
• Check if trigger preset is "on-demand" — in that mode, press Alt+Enter to trigger.
• Open the Output panel ("Bespoke AI") and check for errors.
• Run the "Bespoke AI: Restart Pools" command from the Command Palette.

"Claude Code CLI not found"?

• Install Claude Code: curl -fsSL https://claude.ai/install.sh | bash (macOS/Linux) or npm install -g @anthropic-ai/claude-code (any platform)
• Restart VS Code after installation.

"Authentication required"?

• Run claude in your terminal and follow the login prompts.
• Ensure you have an active paid Claude plan (Pro, Max, Team, or Enterprise).

Explain/Fix/Do says the Claude Code executable was not found (Windows)?

• The commands start Claude Code directly, not through a shell. On Windows this needs the native claude.exe (native installer, ~/.local/bin/claude.exe) or the bundled CLI with node on PATH — the npm claude.cmd shim is not used.

Still not working?

• Check for orphaned processes:
  • macOS/Linux: pkill -f "claude.*dangerously-skip-permissions"
  • Windows: Use Task Manager to end node.exe processes running Claude
• Check for a stale lockfile at ~/.bespokeai/pool.lock and remove it.
• Disable and re-enable the extension.

🔒 Privacy and data flow

What leaves your machine. The text around your cursor (prefix and suffix context) goes to the backend you configured. Commit message generation sends your staged diff; Suggest Edits sends the visible editor text. With the Claude Code backend, requests go through your local Claude Code installation; with the API backend, they go directly to your chosen provider's endpoint (or to Ollama on your machine). Nothing else is sent.

No telemetry. The extension collects no usage analytics and makes no network calls other than to your configured backend — plus the optional trace export below, which you have to point at an endpoint yourself. (The Claude Code CLI, when you use that backend, has its own telemetry settings.)

What stays on your machine, in ~/.bespokeai/ (a folder only your user account can open):

Optional trace export. bespokeAI.trace.otlp.endpoint sends trace records to an OpenTelemetry backend you run or subscribe to (for example Langfuse, or an OTel Collector). It is off by default, pauses while VS Code telemetry is disabled (telemetry.telemetryLevel; VSCodium ships with telemetry off, so there export stays paused until you turn telemetry on), and exports prompt text only if you also turn on bespokeAI.trace.otlp.captureContent. The first export to a non-local host shows a notice saying what will be sent where.

Keys. API keys are stored in your OS keychain via VS Code SecretStorage. Settings that choose a backend, endpoint, model, or key variable can only be set in your user (or remote-machine) settings, never by a repository's .vscode/settings.json.

🛠️ Development

npm install && npm run compile    # Build
npm run watch                     # Watch mode (F5 to launch dev host)
npm run check                     # Lint + type-check
npm run test:unit                 # Unit tests (incl. the replay set)
npm run test:coverage             # Unit tests with v8 coverage
npm run test:quality              # Generate completions for every eval scenario (Layer 1)
npm run test:quality:rescore      # Re-run the deterministic checks over saved eval runs
npm run latency-report            # p50/p90/p95 from your local usage ledger

CI runs on every push and pull request to main: lint, type-check, Prettier, the unit suite with coverage on Ubuntu, macOS, and Windows (the pool IPC is platform-specific), and a VSIX build for pull requests. The unit suite is ~1,540 tests at ~71% line coverage; most of what is uncovered is VS Code wiring in extension.ts. Tests that call real models (test:api, test:quality) need keys or a Claude subscription and run locally, not in CI.

Model choice for code. Dedicated autocomplete products mostly use small models trained for fill-in-the-middle (FIM). This extension sends the same instruction-style prompt to general chat models for both prose and code, because prose is its main use and FIM models are trained mostly on code. For code, a FIM-trained model may well be faster and more accurate; that comparison has not been run yet. Any OpenAI-compatible or Ollama model can be added as a custom preset, and bespokeAI.codeOverride can route code to it.

Tested Models

The quality suite has been run against these models. The reference set for prompt changes is CLI sonnet, xai-grok and openai-gpt-4.1-nano, one per extraction strategy; the others were last run before the 2026-09 prompt change.

¹ Run under the retired id grok-4-1-fast-non-reasoning, which xAI has served since 2026-05-15 as grok-4.3 with reasoning off: the same request this preset now sends. ² The earlier run (2026-03-01) was on grok-code-fast-1, retired 2026-05-15 and now served as grok-build-0.1.

Coverage: all 3 extraction strategies, 6 providers, 3 cost tiers, code-specialized model, local option. See AGENTS.md for testing commands.

🤝 Contributing

Contributions welcome — fork the repo, create a branch, and open a pull request.

• Run npm run check before submitting (must pass)
• See AGENTS.md for architecture details, testing workflows, and coding conventions
• Open an issue for bugs or feature ideas

👤 Author

Built by Trent McNitt — AI developer specializing in agent development, prompt engineering, and full-stack applications.

Available for contract work →