vLLM-Copilot

Your vLLM-served models inside GitHub Copilot and the VS Code Agents window. Built for teams and production. GitHub Copilot provides the familiar chat, tools, and model picker; you provide the model. Requests to your models (prompts, code, context) are never sent to Copilot! Multi-server, multi-user. Full vLLM request control, live observability, strict data residency. Also works with OpenRouter (400+ cloud models, no local infrastructure, many free options), llama.cpp, LM Studio, and Ollama.

For teams running AI on their own vLLM servers for many users, this gives you the professional Copilot integration:

• Data handling: No subscription. No affiliation. No central service. No telemetry. Prompts, code, and company data go only to each model's configured inference server. They are never sent to GitHub Copilot or GitHub. Other extension traffic (model metadata from HuggingFace, curated presets from GitHub) carries no work content.
• Works in the VS Code Agents window: your vLLM models in the new "Open in Agents" agent cockpit, with autonomous sessions, worktree isolation, all of it. Auto-enabled since v1.35.0, nothing to configure, just restart VS Code. See Agents window guide.
• Production vLLM observability: a live dashboard of server availability, queue status, KV-cache usage, TTFT, throughput, per-request token details, and a cumulative token & cost tracker. Your admins and users will know what is going on!
• Multi-server, multi-user by design: each server and model carries its own endpoint, auth, sampling, and token budget. Different teams, environments, or credentials stay isolated and independently managed. But all models are available in the model picker of familiar Copilot!
• Full request control: model modes give you any vLLM parameter, such as thinking effort, sampling, and structured output. A dedicated Output Length dropdown caps response length without a settings edit. Switch both per model from the Copilot picker.
• OpenRouter: add any of ~415 cloud models in a few clicks. Useful for teams without GPU capacity. Real context window, capabilities, pricing, and actual spend (usage.cost) show on the dashboard. See Using OpenRouter.
• Other backends supported: llama.cpp, LM Studio, and Ollama alongside vLLM, each with core features like chat, streaming, tools, personalities, and usage tracking.

For full view expand the details-arrow:

Every vLLM-Copilot feature at a glance. Click to zoom.

If you want to support this work:

Contents

• Quick Start
• What makes this different from BYOK?
• Enterprise & team deployment
• Features
• Servers & backends
• Commands
• Development
• Support

Quick Start

Requirements: GitHub Copilot Chat (you need a GitHub account, but no subscription), plus either a running model server (vLLM, llama.cpp, LM Studio, Ollama) or an OpenRouter API key.

Add a vLLM server (or llama.cpp, ollama, lm-studio)

Let Copilot write your model config for you

Add OpenRouter Model (full provider support, cost tracking and auto selection)

Remote (SSH/WSL/Containers): This extension runs on the remote host (extensionKind: workspace). Install it while connected to the remote window and VS Code places it on the remote extension host automatically. If you installed it locally first, the remote won't pick it up on its own. Install it on the remote explicitly (Extensions view → Install in SSH: … / WSL: … / Dev Container: …).

What makes this different from BYOK?

VS Code's built-in Custom Endpoint (BYOK) handles chat, tools, vision, streaming, and a thinking-effort picker. It also supports a subset of basic inference parameters, such as temperature and top_p, configured for a model.

vLLM-Copilot is for users who want the full vLLM request surface and a richer workflow around it. It lets you define model modes: named configurations you switch between from the model picker. One model can have separate configurations for reasoning, precise coding, and creative work, including their sampling and vLLM-specific request settings.

Enterprise & team deployment

vLLM-Copilot gives companies (and individuals) controlled access to GitHub Copilot through their own inference infrastructure. Operations get live serving metrics, per-model usage and cost tracking. Broken model output is repaired automatically before it reaches your team.

No subscription. No third party receives work content beyond the configured inference server. No affiliation. No central service. No telemetry. GitHub Copilot supplies the familiar chat, tools, model picker, and other interaction features. Prompts, code, and company data go only to the configured inference server, never to GitHub Copilot or GitHub. Other extension traffic carries no work content; it is limited to model metadata, configuration, metrics, and service status.

• Multi-server at scale: add any number of vLLM servers and use their models interchangeably. Each server and model keeps its own endpoint, auth, sampling, and token budget, giving you isolation across teams, environments, or credentials.
• Per-model credentials: every model entry carries its own requestHeaders / auth, so different scopes and keys are managed independently rather than sharing one global key.
• Cost tracking per model: cumulative token and USD spend (Today / Overall), with OpenRouter models preferring their actual reported cost (usage.cost).
• Reliability for daily use: models misbehave, and this extension repairs it (see Robustness below), alongside bounded Retry-After handling, TLS/proxy/cert diagnostics, and chat session cleanup.

Server Dashboard

Open it with the V icon in the activity bar (left sidebar).

The live observability that makes it worthwhile for teams. A native sidebar (no webviews, no extra ports) shows live metrics per server, showing only the rows each backend actually reports: Queue status: running and waiting request counts (or idle) at a glance Expandable metrics: context window, vLLM version, KV cache usage & hit rate, TTFT, output and prefill speed (tok/s) MTP / speculative decoding: acceptance rate, draft depth, proposal count (when active) Last Request Details: per-server node with the most recent request's tokens (input, output, cached, reasoning), timing (TTFT, queue time, generation time), and throughput. Updated immediately after every prompt, not on the poll interval. The Output channel shows the same exact counts after every request: input/output, cached tokens (prefix cache hit %), cache creation tokens, output throughput (tok/s), and speculative-decoding stats. Token Usage and Cost: a model-first tree, one entry per model carrying the price on the collapsed line ($11.51 today and $31.13 total), expanding to Today and Overall token rows. Sub-cent costs keep fine precision; OpenRouter models use their actual reported cost (usage.cost), never summed with rates. Reset Usage is a right-click action. See usage.md for the design. Right-click a vLLM server for the Deep-Dive webview.

Live server metrics with Last Request details: token counts with context/budget percentages, TTFT (reported vs. measured), generation throughput, and queue time.

Robustness

Models misbehave. Instead of letting a bad response reach your team, the extension repairs the stream so they see working output. This is the difference between a demo and a daily driver.

• Auto-continue on empty or truncated responses. Some models (notably Qwen) occasionally return zero tokens or stop mid-sentence. The extension retries with an assistant prefill, so you never see a blank or cut-off answer. Configurable per model (autoContinueRetries, default 1).
• Tool-call repair. When a model truncates a tool call mid-JSON (finish_reason: 'length'), the extension recovers the partial call with jsonrepair + best-effort-json-parser. These are the same libraries Copilot's BYOK uses, so the call is not dropped to empty {}.
• Bounded retries. Transient server errors are retried once, honoring Retry-After (capped at 10 s), and never after partial output has already been streamed.
• Connection diagnostics. Corporate proxy? TLS-inspecting gateway? Missing intermediate certs? Test & Refresh Models verifies servers are reachable, lists loaded models, and corrects ID mismatches. Diagnose Connection runs a deep report comparing SChannel vs. OpenSSL, DNS/TCP reachability, cert-chain inspection, proxy detection, and a settings dump, with a one-line failure classification.

Features

Model Settings

Open it with the V icon in the activity bar (left sidebar); it sits below the Dashboard.

A visual editor for per-model configuration, no settings.json required: Server & model selectors, including unconfigured models Auto-Configure re-runs preset/HuggingFace discovery (modes, capabilities, token budgets, defaults) Sectioned layout: General, Token Budget, Capabilities, Request Params, Transport, Model Modes, System Prompt Parameter picker with known params and type hints; enum and boolean params render as dropdowns Remove Model, Revert, and auto-refresh on config changes

Edit model configuration in a visual editor. No settings.json required.

Model modes are named parameter presets you switch between from the Copilot model picker, like profiles for different tasks: deep reasoning, precise coding, creative work. Each mode merges its parameters into the vLLM request on top of the model defaults. Thinking toggles: enable_thinking: true/false (from bundled presets) Sampling presets: temperature/top_p combinations for creative vs. precise output Structured output: JSON schema enforcement for data extraction Anything vLLM supports: bad words, repetition detection, request shaping Output length is its own control. Models and presets whose maxOutputTokens is an array of response lengths (bundled presets do) get a second, independent Output Length dropdown next to the mode picker (e.g. 16K / 32K / 64K), so you cap response length per request without touching settings. VS Code remembers your pick per model, and a shorter pick hands the freed tokens back to your prompt: Copilot's input budget grows to match. Right after the dropdown first appears for a model, VS Code's picker can lag and show only the mode section. Open the model list once and click the Output Length chip on the model, which fixes it (see screenshot; VS Code snapshot bug, reported upstream). Add vLLM Server & Model auto-generates modes from bundled presets (or OpenRouter reasoning metadata). An example config and the full syntax are in the Manual → Model modes.

Switch between model modes directly from the Copilot model picker: Think, No Think, Precise, etc. Dropdown not showing right after an update? Open the model list and click the Output Length chip (arrow). One click heals VS Code's stale picker snapshot.

Hidden System Instructions (capture & replace). Copilot injects ~21KB of hidden safety and identity rules into every request. Capture them, then replace them:

1. Set vllm-copilot.systemMessageCapture: true
2. Unique system messages are written to .vllm/system-messages.json
3. Write a JSON file of find/replace rules
4. Point systemMessageReplacementsFile at it on the model entry

Replacements are exact substring matches, applied sequentially to every system message before it reaches vLLM.

Workspace custom instructions. .github/copilot-instructions.md, AGENTS.md, and CLAUDE.md are merged into the system message, the same way VS Code handles workspace-level custom instructions.

Personalities & System Prompts

This uses the System-Instructions-Replacement of above with pre-made instructions. Two ways to control what the model sees as its system prompt.

Personality presets replace Copilot's ~21KB of system-prompt boilerplate with something useful. Pick one per model (Model Settings → General, or Set Model Personality); Default restores the original. The choice follows you across workspaces.

Bundled presets are extension-owned and re-synced on every apply; custom behavior belongs in your own replacement file (below) or a user-created personality.

Chat Session Cleanup

Copilot accumulates session data across workspaces. Clean Copilot Sessions lets you pick which workspaces to wipe when sessions grow stale. Access via Ctrl+Shift+P → Clean Copilot Sessions (under Utilities).

Servers & backends

vLLM request controls

Beyond the basics, vLLM request-body parameters give you full request control: structured_outputs (JSON / regex / grammar constraints), bad_words, repetition_detection, chat_template_kwargs (enable_thinking, preserve_thinking), thinking_token_budget, stop-token control, min_tokens, allowed_token_ids, and more. Configure them in defaultParams (model-wide) or modelModes (switchable from the picker). The BYOK Custom Endpoint exposes none of these. Every parameter, its range and backend support: configuration reference.

Supported servers

vLLM is the primary target and gets the full feature set. llama.cpp, LM Studio, Ollama, and OpenRouter are supported alongside it with the core features.

The backend is auto-detected on Add Server and in Model Settings; set it explicitly per model via serverType.

Every backend gets: native Copilot integration (chat, tools, vision, streaming), model modes, output length picker, personality presets, hidden-system-prompt capture & replace, per-server auth/sampling/token budget, auto-continue on empty responses, token usage & cost tracking, and Test & Refresh / Connection Diagnostics.

vLLM-only: vLLM-specific request parameters, per-request server metrics (TTFT/TPOT, KV cache, speculative decoding), and the Deep-Dive webview. Other backends show client-measured throughput instead; the dashboard shows only the rows each backend actually reports and resolves the model's context window per backend.

Using OpenRouter

OpenRouter is a managed remote: no server to run, nothing to install. Add one fixed endpoint and pick from ~415 cloud models. Setup is covered in Quick Start above. A :free suffix is a routing variant and always stays on the free model. Full guide, manual config, and attribution headers: OpenRouter guide.

Server Deep-Dive (vLLM)

Right-click a vLLM server node → vLLM Deep-Dive opens a per-server webview with the full metric set: histogram breakdowns (TTFT/TPOT/token counts) as hoverable bars and the raw metric dump. vLLM-only, since non-vLLM backends don't expose /metrics. It takes one reading when it opens and then stops polling — run the command again to retake it.

Full per-server metrics with live polling, histogram breakdowns, and raw metric dump.

Commands

The full command list, including auth rotation, model/server removal, log tools, and session cleanup: Manual → Commands.

Development

Dev setup, project rules, and the dependency license policy (permissive-only allowlist enforced by npm run license:check in the build, THIRD-PARTY-NOTICES.txt regenerated with npm run license:notices): CONTRIBUTING.md.

Support

If vLLM-Copilot saves you time, money, or your sanity, consider fueling the caffeine habit behind it:

• PayPal: paypal.me/DieterSchwarzmann
• Ko-fi: ko-fi.com/fuzzifikation
• GitHub Sponsors: github.com/sponsors/fuzzifikation

Every donation is appreciated, even a coffee. It keeps local AI development free and open source.

More details

This page covers the essentials. Every setting, parameter, and backend detail is in the vLLM-Copilot Manual.

Your vLLM servers are not limited to VS Code: the GitHub Copilot CLI terminal agent runs on them too, via three environment variables. And since v1.35.0 your models work in the VS Code Agents window ("Open in Agents"), enabled automatically with nothing to configure.

License

MIT License

Copyright (c) 2026 Systemwissenschaften TGU, TTI GmbH

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.