What makes this different from BYOK?

VS Code's built-in Custom Endpoint (BYOK) handles chat, tools, vision, streaming, and a thinking-effort picker. It also supports a subset of basic inference parameters, such as temperature and top_p, configured for a model.

vLLM-Copilot is for users who want the full vLLM request surface and a richer workflow around it. It lets you define model modes — named configurations you switch between from the model picker. One model can have separate configurations for reasoning, precise coding, and creative work, including their sampling and vLLM-specific request settings.

vLLM-specific request controls

These controls are sent as vLLM request-body parameters. Configure them in defaultParams for a model-wide default, or in modelModes to make them switchable from the model picker. They are not exposed by the BYOK Custom Endpoint.

Quick Start

Prerequisites: A running vLLM server (any OpenAI-compatible endpoint) + GitHub Copilot.

1. Install from the VS Code Marketplace
2. Add a model: Ctrl+Shift+P → Add vLLM Server & Model → enter your server URL → pick a model → done. The extension auto-configures everything (model family, thinking modes, context window) from bundled presets or HuggingFace.
3. Change the personality (optional): Ctrl+Shift+P → Set Model Personality → pick your model → pick a personality → done. Four bundled presets replace Copilot's boilerplate with something actually useful.
4. Chat: Open Copilot Chat, pick your model from the dropdown. Switch modes from the same picker.

Remote (SSH/WSL/Containers): This extension runs on the remote host automatically when installed from the Marketplace. VS Code will install it on the remote extension host.

Everything is per-model. There is no global server or global sampling. Each model entry carries its own serverUrl, requestHeaders, and params. Settings take effect immediately — no reload needed.

Features

Model Modes — switchable configurations per model

Model modes let you define named configurations for a model and switch between them from the Copilot model picker — like having "profiles" for different tasks. Think of them as presets: one for deep reasoning, one for precise code, one for creative brainstorming.

Each mode is a set of parameters merged into the vLLM request. Common use cases:

• Thinking toggles — enable_thinking: true/false (the extension auto-detects these from HuggingFace)
• Sampling presets — different temperature/top_p combinations for creative vs. precise output
• Structured output — JSON schema enforcement for data extraction
• Anything vLLM supports — bad words, repetition detection, token budgets

"modelModes": {
  "Think": {
    "chat_template_kwargs": { "enable_thinking": true, "preserve_thinking": true },
    "temperature": 1.0, "top_p": 0.95
  },
  "No Think": {
    "chat_template_kwargs": { "enable_thinking": false },
    "temperature": 0.7, "top_p": 0.8
  },
  "Precise": { "temperature": 0.1, "top_p": 0.1, "top_k": 20 }
}

The Add vLLM Server & Model command auto-generates modes from bundled presets or HuggingFace data. For existing entries, see the Configuration Reference.

Personality Presets

Four bundled personalities that replace Copilot's 21KB system prompt boilerplate with something actually useful. One command, no JSON editing:

1. Set Model Personality: Ctrl+Shift+P → pick your model → pick a personality → done

Want to customize? Copy any preset to .vllm/my-replacements.json, edit the replace fields, point your model at your copy.

Hidden System Instructions (Capture & Replace)

Copilot injects hidden instructions into every request (~21KB of safety rules, identity instructions). This extension lets you capture and surgically modify them:

1. Enable capture — set vllm-copilot.systemMessageCapture: true in settings
2. Inspect — unique system messages are written to .vllm/system-messages.json
3. Create replacements — write a JSON file of find/replace rules
4. Apply — set systemMessageReplacementsFile on your model entry

Replacements are exact substring matches, applied sequentially to every system message before it reaches vLLM. Matched rules are logged in the capture file.

Auto-Continue on Empty Responses

Some models (notably Qwen) occasionally return zero tokens or truncated output. The extension automatically retries with an assistant prefill — you never see a blank or cut-off response. Configurable per-model (autoContinueRetries, default: 1).

Token Usage & Performance Stats

After every request, the Output channel shows exact token counts from vLLM:

• Input/output tokens, cached tokens (prefix cache hit %)
• Output throughput (tokens/sec), time-to-first-token
• Speculative decoding stats (accepted/rejected predictions)

Connection Diagnostics

Corporate proxy? TLS-inspecting gateway? Missing intermediate certs?

• Test & Refresh Models — verify all configured servers are reachable, lists loaded models, corrects model ID mismatches, checks VS Code network gating settings
• Diagnose Connection — deep diagnostic comparing SChannel (PowerShell) vs. OpenSSL (Node fetch), DNS/TCP reachability, cert chain inspection, proxy detection, VS Code settings dump. One-line classification of the failure.

Tool Calling & Truncated Response Recovery

When vLLM truncates a tool call mid-JSON (finish_reason: 'length'), the extension uses jsonrepair + best-effort-json-parser (the same library Copilot's BYOK uses) to recover partial content instead of silently dropping it to empty {}.

Workspace Custom Instructions

The extension merges .github/copilot-instructions.md, AGENTS.md, and CLAUDE.md into the system message — the same way VS Code handles workspace-level custom instructions.

Legacy Configuration Migration

Upgrading from an older version? The extension auto-migrates legacy global server/sampling settings into per-model entries on first launch. One-time, idempotent, no data loss.

Chat Session Cleanup

Copilot accumulates session data across workspaces over time. The Clean Copilot Sessions command lets you pick which workspaces to wipe — useful when sessions grow stale or you want a fresh start. Access via Ctrl+Shift+P → Clean Copilot Sessions (under Utilities).

Commands

Utilities (maintenance, not daily workflow):

Configuration Reference

See Configuration Reference for the complete guide including:

• Model entry fields (all fields, defaults, descriptions)
• Parameters for defaultParams and modelModes (full param table)
• Full JSON syntax reference (every supported field)
• Multiple servers with isolated auth
• System message replacements (detailed guide)
• Personality presets (all 4 presets with descriptions)
• Diagnostics settings
• Troubleshooting (commands table, TLS, corporate networks, Diagnose Connection)
• Known limitations

Quick minimal config

"vllm-copilot.models": [
  {
    "id": "Qwen/Qwen3.6-27B-FP8 on localhost:8000",
    "vllmModelId": "Qwen/Qwen3.6-27B-FP8",
    "serverUrl": "http://localhost:8000"
  }
]

Run Add vLLM Server & Model to generate this automatically. It auto-detects family, max_model_len, capabilities, and applies bundled presets.

License

MIT

MIT