LiteLLM Provider for GitHub Copilot Chat

Use 100+ LLMs in VS Code with GitHub Copilot Chat powered by LiteLLM.

Features

• 100+ LLMs through a unified API (OpenAI, Anthropic, Google, AWS, Azure, Ollama, and any OpenAI-compatible endpoint)
• Multi-server support: connect to multiple LiteLLM servers simultaneously and aggregate models
• Streaming chat completions with SSE
• Function calling (tool use) support
• Multimodal input (vision/image attachments)
• Reasoning/thinking tokens (if the model supports them) — including turning thinking OFF entirely for reasoning models (faster responses, no reasoning tokens burned)
• Dashboard panel for servers, models, and settings
• Document reranking through a dedicated JSON-editor command and a consent-gated Copilot agent tool (not a chat model)
• API key encryption via VS Code SecretStorage
• Per-model pricing display in the model picker
• Status bar context-window indicator — a live fill gauge showing how much of the current model's context window is in use, fed by real backend usage (when reported) or token estimates. VS Code's own "Session Info" panel stays at 0 for custom providers (no host API to feed it), so this is the extension's own equivalent.

Requirements

• VS Code 1.99.0 or higher, with the GitHub Copilot Chat extension installed and signed in
• LiteLLM proxy running (self-hosted or cloud)
• LiteLLM API key (if required by your setup)

Quick Start

1. Install the extension
2. Open VS Code's chat interface (Ctrl+Alt+I / Cmd+Ctrl+I, or the chat icon in the title bar)
3. Run the command "LiteLLM: Add Server" from the Command Palette
4. Enter a label (e.g. Local), base URL (e.g. http://localhost:4000), and API key
5. Back in chat, pick one of the new models in the model picker and send a message

You can also declare the server in settings.json:

"litellm-vscode-chat.servers": [
	{ "label": "Local", "baseUrl": "http://localhost:4000", "apiKey": "sk-..." }
]

The API key is automatically migrated to VS Code's encrypted secret storage on first activation.

Commands

Document reranking (0.2.37)

Configure your LiteLLM server normally; reranking reuses its existing SecretStorage API key. For example, the configured base URL https://proto-llm.gcpdev.alza.cz/llm/ sends rerank requests to /llm/rerank. A terminal /v1 or /v2 is removed for this canonical LiteLLM endpoint; other proxy prefixes are preserved. No additional credential or backend configuration is needed.

Use the exact public model ID registered on your gateway, in this deployment Qwen/Qwen3-Reranker-8B, not a deployment UUID or a renamed example alias. Explicit model mode metadata takes precedence; when /models omits metadata, names containing rerank are inferred as rerankers. The dashboard and Show Available Models retain non-chat and disabled models, but only enabled chat models appear in Copilot's chat picker. Embedding and rerank models cannot handle chat messages.

Manual command

1. Open a JSON document (or select just its JSON), then run LiteLLM: Rerank Documents. With no editor content, the command opens an untitled template: fill it and run again.
2. Select an enabled reranker. The dashboard's Rerank documents action targets that row's server/model instead.
3. If query is omitted, enter it in the input box. top_n defaults to the number of documents and must be an integer from 1 through that count.
4. A cancellable progress notification runs one request. Results open as untitled JSON, sorted by descending relevance_score with original zero-based index and original input document. Nothing is written to disk by the command.

{
       "query": "Which document explains encrypted API key storage?",
       "documents": [
              "VS Code SecretStorage stores server API keys securely.",
              "The dashboard shows model pricing and capabilities."
       ],
       "top_n": 2
}

A JSON string array is also accepted; the command then asks for the query. Only the captured active editor/selection is used, never a workspace scan. Do not supply sensitive content unless you intend to send it to the configured server.

Copilot agent tool

Enable the Rerank supplied documents with LiteLLM tool in the agent's tools picker, or reference #rerankDocuments in a prompt. The registered tool name is litellm_rerank. Supply query, documents, optional top_n, optional exact model (Qwen/Qwen3-Reranker-8B), and optionally server (configured label) or serverId. Omitted model/server selectors work only when exactly one enabled reranker matches; ambiguity returns instructions to specify selectors.

The tool validates input and asks for confirmation explaining that query/documents go to a configured LiteLLM server before any discovery or rerank network request. Model resolution happens after confirmation. Confirmation is managed by VS Code (including its user-controlled tool approval settings). The tool returns JSON to the agent; documents are untrusted data, not instructions. Tool availability/invocation depends on Copilot agent mode and tool settings. This does not intercept or automatically rerank Copilot context: the agent must explicitly call it with candidate strings.

Queries, documents, backend response bodies and credentials are never logged by reranking. Requests use the currently resolved configured key, have no automatic retries, reject redirects, and use chat.timeout as a total response timeout (unlike chat's idle timeout). Responses larger than 8 MiB, invalid scores, duplicate/out-of-range indexes, or malformed JSON are rejected with content-free errors. Cancelling stops local waiting/transport, but cannot guarantee that backend computation or billing stops. Rerank requests do not produce chat session or usage-telemetry events.

Settings

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    VS Code Chat Host                         │
│  (calls LanguageModelChatProvider interface)                 │
└──────────────────────────┬──────────────────────────────────┘
                           │
                ┌──────────▼──────────┐
                │  LiteLLMChatModel    │
                │  Provider            │
                │  (src/provider/)     │
                └──────────┬──────────┘
                           │
           ┌───────────────┼───────────────┐
           │               │               │
    ┌──────▼──────┐ ┌──────▼──────┐ ┌──────▼──────┐
    │  Discovery  │ │  ChatClient │ │  TokenCount │
    │  (/models)  │ │  (/chat)    │ │  (estim.)   │
    └──────┬──────┘ └──────┬──────┘ └─────────────┘
           │               │
    ┌──────▼───────────────▼──────┐
    │     LiteLLM Proxy Server    │
    │  (OpenAI-compatible API)    │
    └────────────────────────────┘

Installation

From VSIX file

Option 1 — VS Code GUI:

1. Open VS Code
2. Open the Extensions panel (Ctrl+Shift+X)
3. Click the ⋯ menu (top-right) → Install from VSIX...
4. Select the litellm-vscode-chat-0.1.0.vsix file

Option 2 — Command line:

code --install-extension litellm-vscode-chat-0.1.0.vsix

From source (development)

npm install
npm run compile

Press F5 to launch the Extension Development Host.

Packaging

To build a .vsix package from source:

npm run compile
npx vsce package --no-dependencies

This produces litellm-vscode-chat-0.1.0.vsix in the project root.

To rebuild after code changes, re-run the same commands — the .vsix is regenerated with the updated code.

Privacy

Your prompts and completions travel only between VS Code and the LiteLLM servers you configure. No telemetry, no third-party calls.

The extension's output log never contains conversation content: request diagnostics use a content-free fingerprint (message counts, role sequence and a one-way hash of user-turn lengths), and server error messages are reduced to a short stable classification (e.g. rate_limit, authentication) instead of being logged verbatim.

Local usage telemetry (optional, off by default)

Setting litellm-vscode-chat.telemetry.localUsage.enabled (machine scope, default false) records one JSONL event per physical HTTP attempt the provider makes — useful for auditing token spend, retry storms and gateway behaviour. It is purely local: nothing is uploaded, and when the setting is off no file or directory is ever created.

Where: <extension globalStorage>/usage-telemetry/v1/YYYY-MM-DD.jsonl (one file per UTC day; files older than 30 days are removed at startup; schema is append-only and versioned via schemaVersion).

What is recorded (metadata only):

• request identity: logicalRequestId (per Copilot request), attemptId + httpAttempt (per physical fetch), retry dimensions (truncationAttempt, networkRetry, cause);
• model: requested model, response model, LiteLLM model id/group;
• response: x-litellm-call-id, response id, HTTP status, outcome (completed / http_error / network_error / timeout / cancelled / stream_error / incomplete_stream / length_limit), finish reason, stable error type;
• timing: start timestamp, duration, time-to-first-byte, time-to-first-token, server-reported durations;
• usage: prompt/completion/total tokens plus cached_tokens, cache_write_tokens, reasoning_tokens, prediction-audio tokens when the backend reports them (missing values stay null, never 0; reasoning tokens are a subset of completion tokens and are never double-counted);
• cost: only when the LiteLLM gateway reports it via allowlisted x-litellm-response-cost* headers, with an explicit cost.source — estimates are never presented as measured costs;
• gateway metadata: x-litellm-attempted-retries / -fallbacks / -max-fallbacks / x-litellm-version;
• validation warnings (e.g. missing terminal usage chunk, mismatched totals).

What is NEVER recorded: prompts, messages, response or reasoning text, tool names/arguments/output, API URLs, server labels, authorization headers or API keys, workspace/user/file paths, raw error bodies, or full header dumps. Each request body also carries only privacy-safe correlation ids (metadata.litellm_vscode_request_id / litellm_vscode_attempt_id / litellm_vscode_client_version) so server-side spend logs can be joined to a client attempt without any content.

Correlating with LiteLLM spend logs: match the event's response.litellmCallId against the server's x-litellm-call-id (preferred), or the request metadata ids above.

License

GPL-3.0-or-later — see the LICENSE file for the full text. AlzaTOP, the sole copyright holder, licenses all versions of this extension, including every previously released version, under the GNU General Public License v3 or later. Copies that were already received under the MIT license keep the rights that license granted them.