U Router
Route LLM requests to cheaper models based on project context, prompt features, and session intent. This extension runs the Python router as a local sidecar and gives you commands, a status bar, and a dashboard.
What this extension does and does not do
It does: run a proxy on 127.0.0.1 that inspects a prompt you send it, decides which of your own configured models should handle it, and forwards the request to that provider using an API key you supply. It records the decision locally in SQLite so you can see what you spent versus a single-model baseline.
It does not: ship any API key, credential, token, or account. It does not provide, resell, or unlock access to any AI model or paid service. It contacts no endpoint you have not configured yourself. With no API key set, it returns a local mock response so you can inspect routing decisions offline.
All network destinations are ones you choose: api.openai.com and api.anthropic.com when you set the corresponding key, plus any OpenAI-compatible endpoint you point it at. Model prices in models.yaml are published list prices used purely for the extension's own cost arithmetic.
How routing works
The router scores each request across four signal families (prompt shape, project context, current-turn intent, decayed session pressure), maps the score to a tier (cheap / mid / frontier), and picks a concrete model from a ranked list. Everything is logged to SQLite with counterfactuals so you can see what a different tier would have cost.
This extension:
- Starts
uvicorn app.main:app as a child process and keeps it healthy.
- Shows a status bar indicator; click it to open the dashboard.
- Provides commands to reload rules, run the weekly catalogue refresh, toggle shadow mode, and report outcomes for the tuner.
- Sends an initial project scan to the router on activation.
- Embeds the dashboard in a webview.
Requirements
- Python 3.10+ available on
PATH (or point llmRouter.pythonPath at it).
- The router source and its rules are bundled with the extension — no separate clone needed.
- First-run dependency install. On first activation the extension shows the exact
pip install -r requirements.txt command needed for fastapi, uvicorn, httpx, pydantic, and pyyaml. Click Open Terminal and the command is pre-typed for you — you review it and hit Enter. Nothing is installed silently. Once installed, click I already installed them and this prompt won't appear again for that Python interpreter.
- If you want to use a different router tree (e.g. a fork you're editing), set
llmRouter.routerPath explicitly and it takes precedence over the bundled copy.
Settings
| Setting |
Default |
Purpose |
llmRouter.pythonPath |
python |
Python interpreter used to launch uvicorn. |
llmRouter.routerPath |
(auto) |
Absolute path to the router source directory. |
llmRouter.port |
8080 |
Local port the router listens on. |
llmRouter.autoStart |
true |
Start the router on activation. |
llmRouter.shadowMode |
false |
Log-only; forward requests to the baseline. |
llmRouter.baselineModel |
claude-opus-5 |
Baseline for savings math. |
llmRouter.rulesUrl |
"" |
Remote config poll URL. |
llmRouter.rulesRefreshSec |
300 |
Remote poll interval. |
llmRouter.weeklyUpdate |
false |
In-process weekly catalogue refresh. |
llmRouter.env |
{} |
Extra env vars (e.g. OPENAI_API_KEY). |
llmRouter.userModels |
{} |
Per-tier model override map. |
Commands
- LLM Router: Start / Stop / Restart
- LLM Router: Open Dashboard
- LLM Router: Reload Rules —
POST /admin/reload
- LLM Router: Run Weekly Update Now —
POST /admin/weekly-update
- LLM Router: Toggle Shadow Mode
- LLM Router: Report Outcome for a Request — feeds
POST /api/outcome, which is what the tuner learns from.
- LLM Router: Show Output
Using the router
Point any OpenAI SDK at http://127.0.0.1:<port>/v1. Attach an X-Router-Session header if you want session-pressure tracking across turns.
Development
npm install
npm run compile
# In VS Code: F5 to launch an Extension Development Host.
Package a .vsix:
npm run package
Publish (requires a Marketplace publisher ID and PAT):
npm run publish
License
MIT.
