Model Gate

An LLM gateway inside VS Code: it lets Claude Code, Codex and GitHub Copilot use local models (Ollama, LM Studio, llama.cpp) or any cloud model.

Everything goes over HTTP so that usage can be made visible. All three clients pass through a single gateway, so tokens and cost compare directly under one set of definitions.

Claude Code ──> /v1/messages          (Anthropic format)  ┐
Codex       ──> /v1/responses         (Responses format)  ├─> translate ─> any provider
Copilot     ──> /api/chat, /v1/...    (Ollama/OpenAI)     ┘             └─> usage record

Install

For now the .vsix is distributed directly.

code --install-extension model-gate-0.1.0.vsix

From the GUI: the Extensions view, … in the top right, "Install from VSIX…". It needs VS Code 1.90 or later and Node.js 20 or later.

Using it

1. Start it (no configuration)

Enabling the extension starts the gateway automatically. At the same time it:

• probes 127.0.0.1:11434 (Ollama), :1234 (LM Studio), :8080 (llama.cpp), :8000 (vLLM)
• publishes whatever models it finds (fetching context length and tool support from the upstream)
• generates an authentication token and stores it in SecretStorage

No configuration file is required at this point.

2. Connect the clients

Run Model Gate: Set Up Clients from the command palette. Each client gets its own token, so usage is reliably attributed per client.

The pre-write preview opens as a read-only virtual document. An unsaved Untitled tab would prompt to save on every close, and an accidental save would drop a file somewhere unrelated. The tab closes itself once you have answered (write / cancel).

Claude Code (the VS Code extension)

The Claude Code extension inside VS Code does not inherit terminal environment variables. export ANTHROPIC_BASE_URL=... will not connect, so the settings go into the env block of settings.json. There are three places to write it:

What gets written:

{
  "env": {
    "ANTHROPIC_BASE_URL": "http://127.0.0.1:4000",
    "ANTHROPIC_AUTH_TOKEN": "mg-claude-code-...",
    "CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY": "1"
  },
  "availableModels": ["claude-fable-5", "claude-opus-5", "gpt-5.6-sol", "FW-Kimi-K3"],
  "enforceAvailableModels": true,
  "model": "claude-fable-5"
}

availableModels lists the published models (their ids) verbatim. Setting enforceAvailableModels means the model picker only offers models that really are callable through the gateway, so you cannot pick a name that fails. When no models are published, neither key is written, to avoid an empty picker.

Files carrying an authentication token are made readable by their owner only (600). A write-temp-and-rename scheme otherwise takes on the new file's mode, so the original mode is inherited and, for secrets, group and other read access is dropped (measured: a config.toml Codex had created with 600 had loosened to 644).

Existing settings such as permissions and hooks survive. Rather than loading and writing back, only our own keys are changed, as a diff — comments, ordering and indentation stay as they were (positions are located with jsonc-parser, the same library VS Code uses for settings.json, while the formatting is matched to the original file here). Writing and then removing restores the original text. The content is always previewed and confirmed before saving. If .claude/settings.local.json is not in .gitignore, adding it is offered.

The same menu has "Remove the Model Gate settings" to undo it (deleting the three env keys plus availableModels / enforceAvailableModels, leaving everything else). Only files Model Gate wrote are touched, and within them only the keys it wrote (model stays — it may be the user's own choice). A file counts as a target when ANTHROPIC_BASE_URL points at a loopback address or ANTHROPIC_AUTH_TOKEN starts with mg-. availableModels is also a genuine Claude Code setting, so it alone is never taken as evidence of "Model Gate settings" — that is what stopped a hand-written settings.json from being destroyed as collateral. The target files are listed for confirmation before anything is deleted. Settings left pointing at a stopped gateway make Claude Code unusable, so removal always exists. In particular, restoring the endpoint while leaving enforceAvailableModels behind would leave no selectable model, so both go together.

Codex

This is what gets written:

model = "<default model>"
model_provider = "modelgate"

[model_providers.modelgate]
name = "modelgate"
base_url = "http://127.0.0.1:4000/v1"
wire_api = "responses"                        # the only value Codex accepts
experimental_bearer_token = "mg-codex-..."    # the key goes here, inline
requires_openai_auth = false

model_context_window = 1000000                # so limits are handled correctly (see below)

• wire_api can only be responses. The Codex configuration reference states "responses is the only supported value", and chat does not connect. That is why the gateway also accepts POST /v1/responses (see the protocol table below).
• The key is written inline and env_key is deleted. env_key is the field for the name of an environment variable; putting the key itself there gives Missing environment variable: mg-.... Measured further: with both env_key and experimental_bearer_token present, env_key wins, and it fails on any machine where that variable is unset. So env_key is removed on write (MODELGATE_API_KEY is also set in VS Code's terminals, so an env_key style still works if you prefer it).
• experimental_bearer_token is experimental, as the name says. Codex may rename it.
• Model metadata for '...' not found comes from the published name. Codex looks up its built-in model list by name, so a name it does not know produces this (a display-only warning; nothing stops working). Measured: with the same Azure claude-opus-5 behind it, publishing it as gpt-5.6-sol made the warning disappear. If it bothers you, rename the model to something Codex knows — but then the name and the substance disagree, which is why the usage panel lets you see the upstream model that actually answered (requested, resolved and upstream names are recorded separately). model_context_window has nothing to do with the warning; it is written so context length is handled correctly.

Verified against real hardware (Codex CLI 0.147.0):

$ codex exec "run echo hello-from-codex and tell me the output"
  exec /bin/bash -c 'echo hello-from-codex' succeeded in 0ms: hello-from-codex
  codex The output is `hello-from-codex`.
  tokens used 27,731

The full round trip — tool call, execution, result returned, conversation continued — works.

Asking Codex "who are you?" answers "Codex CLI, provided by OpenAI…". That is Codex sending its own system prompt; it says so regardless of what it is connected to. Which model actually answered is visible in the usage panel (the upstream model column). For comparison, the same route and model asked without that system prompt:

$ curl .../v1/responses -d '{"model":"claude-opus-5","input":"which company built you, and which model are you?"}'
I am Claude, an AI built by Anthropic.

Pinning a model per workspace

Codex layers <workspace>/.codex/config.toml on top of the user-level file, so the default model can differ per project. The setup flow offers it as a second target, alongside "this machine".

Two measured limits shape what that file may contain.

• model_provider and model_providers are refused there. Codex says so out loud: Ignored unsupported project-local config keys in <ws>/.codex/config.toml: model_provider, model_providers. If you want these settings to apply, manually set them in your user-level config.toml. A sensible refusal — otherwise a cloned repository could redirect Codex, and the token it carries, to an endpoint of its choosing. So the endpoint always stays in ~/.codex/config.toml, and choosing the workspace target writes there too (leaving the machine-wide model alone, since that is the user's own choice).
• A project-local configuration is ignored until the directory is trusted. Codex asks once per folder ("Trusting the directory allows project-local config, hooks, and exec policies to load"). Before that it is silently skipped, so the notification after writing says as much.

What lands in the workspace is therefore just:

model = "claude-opus-5"
model_context_window = 1000000

No endpoint, no token — so unlike .claude/settings.local.json, this file can be committed and shared with the repository.

When a configuration already exists, you get the same two choices as with Claude Code.

Both are differential edits. Approval settings, other providers, MCP server definitions and comments are untouched (and the content is previewed before writing). Removal drops [model_providers.modelgate] and model_provider, and deletes model and model_context_window only when the model is one this gateway publishes (the user may have named a different model; conversely, leaving our model name while removing the endpoint would make Codex send OpenAI a name it does not know, and fail).

3. Look at the usage

Click $(server) Model Gate :4000 · 1.2M tok in the status bar and choose Open Usage. It shows tokens, cost, latency, TTFT and errors by client, by model and by provider, with CSV/JSON export.

Daily heatmap

At the top of the panel, daily token counts are laid out like GitHub's contribution graph, covering a full year (53 weeks x 7 weekdays). It is not affected by the period filter — it is a time axis itself, so shrinking it to one cell for "Today" would be pointless.

Intensity comes from quartiles of the non-zero days, not fixed thresholds. Someone doing 10k tokens a day and someone doing 10M a day use the same extension; with fixed cut-offs one of them gets an entirely dark grid and the other an entirely light one, and the shading carries no information.

The colours are a single-hue lightness ramp (inverted on dark backgrounds), and all four steps have verified contrast against the background. Hovering or focusing a cell shows the numbers, but nothing depends on colour and hover alone: the same numbers are readable as a table.

Per-model connection test

The model list in the settings panel has a Test button per row and a Test all button, because a reachable endpoint does not mean a given model is callable:

• Azure answers 404 DeploymentNotFound for catalogue names that were never deployed
• permissions, billing and region limits can differ per model
• some models have no /chat/completions and are callable only through the Responses API

So instead of a list fetch or a HEAD, it sends one minimal max_tokens: 16 request through the same provider implementation the gateway uses in production. Dialect handling and automatic delegation to the Responses API take the same path, which preserves the guarantee that whatever passes here passes from a client. It consumes a few tokens.

Failures are turned into sentences with a cause (404 DeploymentNotFound becomes "check the deployment name", 429 becomes "rate limited (the connection itself works)"). Presenting a 429 as a plain failure sends people off to change configuration that was never wrong.

"Test all" runs sequentially. Firing them in parallel trips rate limits, and then a model that cannot be called is indistinguishable from one that was merely throttled. Models you have not tested show a grey "Untested" rather than an error colour.

State colours

Green for healthy, red for failing, everywhere (status bar, sidebar, settings panel).

The status bar judges failure from the most recent results, not today's total error count. Staying red all day over one morning failure makes a working gateway look broken. The icon shape changes as well (● / error / circle-slash), so nothing depends on colour alone.

Configuration

The configuration file is ~/.modelgate/config.yaml (optional). It is designed to minimize how much you have to write.

Adding a cloud provider

providers:
  openrouter:
    key: secret:openrouter   # the value lives in SecretStorage (registered by command)

models:
  - openrouter/anthropic/claude-sonnet-4.5

default: openrouter/anthropic/claude-sonnet-4.5

• Known providers need only a name for type and base_url to be resolved (ollama, lmstudio, llamacpp, vllm, openai, anthropic, openrouter, deepseek, groq, xai, together, mistral, gemini, bedrock, vertex). The exception is vertex, whose base URL contains the project ID, so it needs base_url.
• The settings panel only offers the three big clouds (Azure OpenAI, AWS Bedrock, Google Vertex AI) plus "Other (specify a URL)". The other names above (including gemini) still work if written straight into this configuration file — they are merely hidden from the picker and can be restored at any time.
• context_window, max_output_tokens and pricing never have to be written — they are fetched from the upstream or resolved from the built-in table.
• API keys are never written into the YAML. Store them with Model Gate: Register API Key; the YAML keeps only a secret:<name> reference.
• Setting server.host to anything other than 127.0.0.1, or server.auth to none, produces a warning. This gateway holds cloud API keys and calls upstreams on your behalf, so making it reachable lets everyone who can reach it spend against those keys (the setting itself is allowed).

Registering a model is just typing a name

The settings panel shows no model list. You type a name and register it.

That is because listed names and callable names do not match. What you can call on Azure OpenAI is the deployment name you chose in the portal, while /models returns that region's catalogue (measured: 437 catalogue entries against 11 callable deployments). Sending a catalogue name gives 404 DeploymentNotFound. Conversely, some tenants cannot read the deployment list at all for permission reasons, and deployments exist that the list omits (2 of them, measured).

So a list is either too long or too short, and it guarantees nothing about a name. Put one on screen and it reads as "listed means callable", which also talks people out of correct names that never appear. Fetching it needs extra permissions, too.

Whether a name really works is answered by the per-model connection test, not by a list — it makes a real round trip along the production path, so passing there means passing from a client.

One round trip at registration settles how the model is called, and the answer is stored. Even on upstreams that call themselves "OpenAI compatible", which surface holds the model and which spelling it accepts cannot be known without asking. Discovering that in production would waste a round trip per model and start over after every process restart.

models:
  - id: gemma4
    use: bedrock/google.gemma-4-31b
    route: responses                                                    # <- surface (automatic)
    route_base_url: https://bedrock-mantle.us-east-1.api.aws/openai/v1  # <- only when it differs
  - id: gpt
    use: azure/gpt-5.6-sol
    route: chat
    dialect:
      token_param: max_completion_tokens                                # <- spelling (automatic)

You never have to write these by hand (pressing Test again refreshes them). Measured (registration vs production):

Two things get settled:

• route — which surface holds it (chat / responses / messages / vertex-anthropic)
• dialect — how to spell the request (some deployments reject max_tokens)

When neither is written (hand-written configurations, discovered models), they are learned at runtime as before. For an endpoint with a single surface and no dialect quirks, it is one round trip either way.

"Check connection" on an endpoint only establishes reachability and whether the key is accepted.

Azure OpenAI

• The input is labelled "deployment name". Use the name as listed under "Deployments" in the portal, not the catalogue model name.
• You can paste the portal's resource URL as-is. Entering https://<name>.openai.azure.com appends /openai/v1 automatically.

AWS Bedrock

Bedrock has an OpenAI-compatible endpoint that accepts a Bedrock API key via Authorization: Bearer. No SigV4 signing and no AWS CLI configuration are involved. Choose "AWS Bedrock" in the settings panel and enter a region and an API key (the base URL https://bedrock-mantle.<region>.api.aws/v1 follows from the region).

providers:
  bedrock:
    base_url: https://bedrock-mantle.ap-northeast-1.api.aws/v1   # us-east-1 when omitted
    key: secret:bedrock

models:
  - id: gpt-oss
    use: bedrock/openai.gpt-oss-120b

Things that trip people up:

• There are three surfaces, and models live on different ones. All three sit on the same host. Writing just /v1 in the configuration is enough — the gateway sends models that are not there to the right surface automatically (measured in us-east-1 by calling each of the 55 listed models individually).

  Endpoint	Protocol	Models	Notes
  /v1/chat/completions	OpenAI compatible	38 open-weight models (openai.gpt-oss-*, qwen.*, deepseek.*, mistral.*, zai.* (GLM), moonshotai.* (Kimi), minimax.*, nvidia.*, google.gemma-3-*)	takes max_tokens as-is
  /openai/v1/responses	OpenAI Responses	google.gemma-4-*, xai.grok-4.3, openai.gpt-5.x (*)	rejects max_tokens, so it is rewritten to max_completion_tokens
  /anthropic/v1/messages	Anthropic native	6 anthropic.claude-* models (*)	authenticates with either x-api-key or Bearer

  (*) The account used for the measurements has no entitlement for these two groups (is not available for this account), so a successful round trip is unverified. The connection test reports "not available to this AWS account" in that case — it is not a key problem, so enable access on the AWS side.

  anthropic.claude-* is sent straight to the Anthropic surface based on the ID prefix. Both OpenAI-compatible surfaces refuse it, so trying them only wastes round trips. Other surfaces are settled once at registration and stored, so production requests always take one round trip.

  /v1/responses is not used — sending a model that /v1 refused there produces neither 200 nor 400 and never terminates, so delegation targets /openai/v1 instead.

• Issue API keys in the management console under Bedrock -> "API keys". Short-term keys expire within 12 hours (this is the "it worked yesterday, now 401").

• Model access has to be enabled per region. A model that is not enabled answers 403 AccessDeniedException. That is not a key problem, so the connection test says "enable model access".

• Model IDs are vendor.model (openai.gpt-oss-120b, qwen.qwen3-32b, deepseek.v3.2). On the bedrock-mantle surface there is no region prefix — measured, all 55 IDs this surface returns take that form (adding us. gives a 400). The exact list is available from GET https://bedrock-mantle.<region>.api.aws/v1/models. Inference profile ARNs (arn:aws:bedrock:...:inference-profile/...) can be registered too.

• The same model can appear both dated and undated (openai.gpt-5.5 and openai.gpt-5.5-2026-04-23). Either is fine to write, though being listed still does not mean being callable (as above).

• Delegation is decided from the upstream's refusal. Both model '...' isn't supported on this route and The model '...' does not support the '/v1/...' API are sent to the Responses surface of the other endpoint (/openai/v1). Only when both refuse does it end with "cannot be called on this endpoint". The discovered route is cached per model, so later calls take one round trip.

• API keys are bound to a region. Sending one to another region's endpoint gives 401 Credential should be scoped to a valid region. The key is fine, so the connection test says "the API key belongs to a different region than the endpoint".

Google Vertex AI

Vertex AI's OpenAI-compatible endpoint works with an Express-mode API key (one starting with AQ.). Choose "Google Vertex AI" in the settings panel and enter a project ID and an API key (the URL follows from them).

providers:
  vertex:
    # The project ID goes in the base URL, so this endpoint alone requires base_url.
    base_url: https://aiplatform.googleapis.com/v1/projects/my-project-123456/locations/global/endpoints/openapi
    key: secret:vertex

models:
  - id: flash-lite
    use: vertex/google/gemini-3.5-flash-lite

Things that trip people up:

• It is not an AI Studio key. The "Google Gemini (AI Studio)" option is a different endpoint with a different key. Mixing them up gives a 401.
• Model IDs are publisher/model (google/gemini-3.5-flash-lite). Sending just gemini-3.5-flash-lite gives 400 Malformed publisher model. Only the tail is used as the published name, so clients can call it as something short like flash-lite.
• The location is global. Measured: the same Express key returns 200 on global while us-central1 answered Publisher model ... was not found or your project does not have access. There is deliberately no region picker.
• The key goes only in x-goog-api-key. Sending Authorization: Bearer alongside gives 401 API keys are not supported by this API, so the "send both" approach used for other endpoints does not apply (auth: google-api-key switches this).
• "Check connection" uses no model list. Vertex AI has none, and /models returns 404 HTML whether or not the key is valid. Instead an empty body is posted to /chat/completions, distinguishing 400 (the key was accepted) from 401.

Partner models (Claude and friends) differ in both endpoint and credential

Vertex splits Google's own models from partner models (measured).

Model IDs starting with anthropic/ are routed to the partner-model endpoint automatically (settled at registration as route: vertex-anthropic). The body is Messages format, but model goes in the URL and is not sent; anthropic_version: vertex-2023-10-16 takes its place.

An API key cannot call it. Measured: sending x-goog-api-key gives 401 API keys are not supported by this API, and putting an API key in Bearer gives 401 Request had invalid authentication credentials. Expected OAuth 2 access token (identical on global and us-east5). The connection test reports this as "cannot be called with an API key (an OAuth2 access token is required)". To use it, register the output of gcloud auth print-access-token as the API key (it expires in about an hour).

Unverified: the measurement environment has neither Claude entitlement on the project nor OAuth2 credentials, so a successful round trip has not been confirmed. What is confirmed is that requests reach the right endpoint, the shape of the body, and the error handling. Other partner models (Meta, Mistral, ...) are out of scope: each publisher has its own body format, and the Anthropic shape does not work for them.

When you do want to spell things out

providers:
  mycorp:                              # a provider with no preset
    type: openai-compatible
    base_url: https://llm.example.com/v1
    key: secret:mycorp
    headers: { X-Tenant: dev }

models:
  - id: fast                           # a shorter published name
    use: ollama/qwen3-coder:30b
    max_output_tokens: 32768

aliases:                               # map the fixed IDs Claude Code sends
  "*haiku*": fast

server: { port: 4100, auth: none }

Aliases

Claude Code sends fixed model IDs such as claude-opus-5. Globs map them onto real models. The resolution order is

exact match -> user globs -> built-in globs (claude-*, gpt-*, o3*, ...) -> default

Supported protocols

Inbound (client side)

Outbound (upstream side)

openai-compatible absorbs dialect differences at runtime:

• upstreams that reject max_tokens are switched to max_completion_tokens automatically (measured: Azure's gpt-5.6-sol, AWS Bedrock's /openai/v1)
• Unsupported parameter: 'x' drops that field and retries exactly once (some deployments do not accept reasoning.effort)
• when /chat/completions does not carry the model, it is delegated to the Responses API (Azure's 404 api_not_supported, AWS Bedrock's 400 ... isn't supported on this route. Bedrock serves a different model set per route, so without this some models would be uncallable)
• delegation prefers another surface on the same host. AWS Bedrock has /v1, /openai/v1 and /anthropic side by side, and a model /v1 refused never terminates on /v1/responses while /openai/v1/responses answers (measured). IDs starting with anthropic. go to the Anthropic-native surface (/anthropic/v1/messages), so one endpoint spans three protocols

All of this is established once, at registration, and stored in the configuration as route and dialect (see registering a model). Without it in the configuration, it is learned and cached in process — but that dies with a restart, so settling it at registration is the main path.

Key placement is switched per upstream too (the default sends both Authorization: Bearer and api-key; Vertex AI alone uses x-goog-api-key on its own). See Google Vertex AI.

Stall timeouts

A silent upstream never leaves a client waiting indefinitely.

Measured: for one model, AWS Bedrock's /v1/responses returned response.created and then sent nothing but SSE keep-alives (: lines), with no terminator after seven minutes. Keep-alives are discarded per the specification, so from the gateway it looked completely silent. Previously only the connection test had a deadline and the production path had none, which left clients hanging.

Debug logging

For investigating problems, request and response bodies can be recorded in the output channel. There are four ways to toggle it, and none of them needs a restart.

While it is on, prompt and response contents are retained verbatim. They may contain work code or personal data, so enabling it says so on screen once. API keys and authentication tokens are never logged on any path (headers are not recorded).

Usage records

One line per request is appended to ~/.modelgate/usage/YYYY-MM-DD.jsonl (kept 365 days by default). The retention matches the heatmap window: a shorter one would make "empty because it was deleted" indistinguishable from "empty because it was unused" (change it with usage.retention_days). JSONL rather than SQLite, to avoid a native dependency.

Recorded: client, inbound protocol, requested model, resolved model, provider, upstream model, input/output tokens, cache tokens, measured or estimated, cost, latency, TTFT, status, aborted flag.

Two design contracts:

• Measured and estimated values are never mixed. When the upstream reports no usage, it falls back to estimation, sets tokenSource: 'estimated' and shows it distinctly in the UI.
• Unknown pricing is never treated as zero. Local inference really is free, but a cloud model without pricing is undefined and displays as "cost unknown".

OpenAI-compatible streams report no usage without stream_options.include_usage, so the gateway always adds it upstream (and does not forward a usage chunk downstream when the client did not ask for one).

Commands

Development

npm install
npm run typecheck   # type checking
npm test            # automated tests (273, no external traffic; includes activating the artefact)
npm run build       # produce dist/extension.js (bundling the ESM builds of dependencies)
npm run watch       # watch build

Release steps

npm run typecheck && npm test        # 273 tests (including activating the artefact)
npm run package                      # model-gate-<version>.vsix
unzip -l model-gate-*.vsix            # confirm testcon.txt and .claude are absent
git tag -a v<version> -m "Model Gate <version>"

npm run icon regenerates the icon (media/icon.png, 256x256) via scripts/make-icon.py. It needs no external libraries: the shapes are signed-distance fields and the label is filled straight from a TrueType outline by scripts/ttf.py. Any bold grotesque on the machine will do — the script tries a list of usual paths and tells you if it finds none. Before publishing to the Marketplace, change publisher to a registered ID, set repository to the public URL, and replace the copyright holder in LICENSE with a real name.

Verifying against real endpoints

Place testcon.txt (line 1 base_url, line 2 API key) to run the scripts that exercise real endpoints. That file is gitignored.

npm run compile
node out/test/manual/live.js   # the provider layer only
node out/test/manual/e2e.js    # the whole gateway (3 protocols + separated attribution)

Design notes

Translation is N+M, not N×M. Rather than combining 4 inbound protocols with 3 upstream kinds directly, everything goes through an internal representation (IR). The IR follows Anthropic's block-oriented shape, because OpenAI's flat delta format can be derived from it mechanically while the reverse (flat -> blocks) cannot recover the boundaries.

Three things break most easily in translation (pinned by tests):

1. Where tool_result goes — Anthropic puts it in a block array inside a user message, OpenAI in standalone role: 'tool' messages. N of them in one turn have to expand into or collapse from N messages. Forwarding them split up breaks parallel tool calls.
2. OpenAI's tool_calls[].index — it numbers within the tool_calls array and does not match the IR block index.
3. output_tokens in Anthropic's message_delta is cumulative — adding it up double-counts.

Ollama is NDJSON, not SSE, and does not accept chunked tool calls (arguments are buffered and emitted in one message).