Adaptive Local AI

Adaptive Local AI runs a private, OpenAI-compatible model router inside VS Code or Cursor. Connect Cline, Continue, or another compatible coding client to one stable endpoint and let the extension select a local model for each request.

The first release is a control plane. Your coding client still owns file reads, edits, terminal commands, approvals, and conversation history. The extension owns model selection, streaming proxy behavior, authentication, single-GPU scheduling, model unloads, and metadata-only routing traces.

راهنمای فارسی

What works in 0.1.0

• POST /v1/chat/completions, including Server-Sent Events streaming
• Tool-call and structured JSON pass-through
• GET /v1/models with local-auto, local-fast, local-agent, and an optional local-deep
• English and Persian task signals plus structured client metadata
• Manual route aliases and automatic routing with conversation hysteresis
• A serialized inference queue for a single GPU
• Ollama unload on route switches and after an idle timeout
• A generated gateway key stored in VS Code SecretStorage
• An activity-bar status view, status bar item, setup commands, and logs
• Local JSONL traces that never contain prompt or response bodies
• An optional separate OpenAI-compatible deep backend

Requirements

• VS Code 1.95+ or a compatible Cursor build
• Ollama running locally
• At least one installed coding model
• An OpenAI-compatible coding client extension, such as Cline

Quick start

1. Install the VSIX or Marketplace extension.
2. Open the Adaptive Local AI icon in the Activity Bar.
3. Run Test Ollama Connection.
4. Run Configure Installed Models and choose the fast and agent models.
5. Run Copy Client Configuration.
6. In your coding client, choose OpenAI Compatible and paste these values:

For a direct Ollama baseline before enabling the router, use http://127.0.0.1:11434/v1 and an exact model name from ollama list.

Routes

Automatic routing uses the latest user task, request size, file references, tool availability, and optional structured metadata. A previous route can be retained when a client sends a stable conversation ID and the new score does not clear the configured switch margin.

The strategy is observable metadata in this release. The client continues to own its tool loop. This prevents two independent agent loops from changing the same workspace.

Response metadata

Every response includes routing headers:

• x-adaptive-route
• x-adaptive-model
• x-adaptive-provider
• x-adaptive-strategy
• x-adaptive-reason
• x-adaptive-queue-ms
• x-adaptive-fallback when a fallback occurred

Non-streaming JSON responses also contain an adaptive_local_ai object. A missing deep model can explicitly fall back to the agent model, or return an error, according to adaptiveLocalAI.routing.deepFallback.

Structured routing metadata

Clients can add hints without placing routing instructions in the prompt:

{
  "model": "local-auto",
  "messages": [{ "role": "user", "content": "Investigate the failing request" }],
  "metadata": {
    "adaptive_local_ai": {
      "conversation_id": "issue-142",
      "requires_tools": true,
      "repo_wide": false,
      "changed_file_count": 4,
      "repo_file_count": 2200,
      "latency_sensitive": false,
      "high_risk": false
    }
  }
}

The extension removes metadata.adaptive_local_ai before forwarding the request. A client can also send x-adaptive-conversation-id.

Deep backend and AirLLM

The deep route is disabled until a model is configured. For an Ollama model, set Models: Deep and leave Deep Provider as ollama.

For AirLLM or another runtime, expose it through a separate OpenAI-compatible /v1/chat/completions adapter, then set:

• Models: Deep Provider to openai-compatible
• Models: Deep Base URL to that server's /v1 base URL
• Models: Deep to its exact model ID
• Run Adaptive Local AI: Set Deep Backend API Key if the server requires a key

The extension serializes calls to an external deep server, but that server owns physical model unload. A production AirLLM adapter should expose lifecycle control before sharing the same GPU with Ollama.

Security and privacy

The server binds to 127.0.0.1 by default and requires a randomly generated API key for every endpoint except /health. A non-loopback bind is rejected unless remote access is explicitly enabled. Keys are stored in VS Code SecretStorage and are never logged.

Routing traces contain timestamps, models, scores, reasons, latency, and outcome. They do not contain messages, code, tool arguments, or model output. See PRIVACY.md and SECURITY.md.

Current boundaries

• The API surface is Chat Completions plus model discovery; the Responses API is not implemented yet.
• Vision is disabled in the built-in capability profiles until a model-specific registry is added.
• Model switching occurs between requests. A live SSE stream is never spliced between models.
• The extension does not replace Cursor Tab autocomplete.
• The built-in router starts with deterministic rules. Claims about model quality require task-level benchmarks on the target machine.

Development

npm ci
npm test
npm run package:vsix

Press F5 in VS Code to launch an Extension Development Host. Integration tests use a mock Ollama server and do not require a model download.

See docs/ARCHITECTURE.md, the Persian product roadmap, CONTRIBUTING.md, and PUBLISHING.md.

License

MIT. The implementation is original. Research projects that informed the design are listed in THIRD_PARTY_NOTICES.md; their code and assets are not bundled.