Silo - Local & Cloud AI Coding Assistant

Agentic coding assistant for VS Code. Runs fully local on Ollama or routes to major cloud and OpenAI-compatible AI providers. Your keys never leave your machine.

Current release: 1.5.0.

Why Silo

• Private by default - local Ollama backend. Nothing leaves your machine unless you pick a cloud provider.
• No bundled model - Silo ships with none of its own. The model picker reads whatever you already have pulled in Ollama and lists it automatically, with real parameter size and quantization.
• Auto model selector - toggle in the toolbar routes complex prompts (code, search, analysis) to your largest installed model and short conversational replies to your smallest, based on whatever models you actually have.
• Live thinking stream - for any local model whose Ollama capabilities include reasoning, its tokens appear in real time in a collapsible Reasoning block. Toggle on/off in the model picker.
• Plan mode (Claude Code style) - Think -> explore -> ask clarifying questions with option buttons -> numbered plan -> todo checklist -> "Switch to Auto" button.
• Slash commands with autocomplete - type / for a floating list of commands with arrow-key navigation.
• Markdown rendering - bold, italic, inline code, headings, lists, blockquotes, code blocks with copy button.
• Agentic - reads, writes, edits, runs shell commands, web search, git ops, tracks todos.
• Approval workflow - Ask mode shows a diff preview with Apply/Reject before touching files.
• Explicit context - use @file, @folder, @codebase, and @docs mentions in chat.
• Checkpoints & checks - Auto/Edit creates checkpoint patches and runs detected project checks after edits.
• MCP-ready - connect external tools through .silo/mcp.json.
• Multi-provider - pick a cloud model per chat. API keys are stored in VS Code's encrypted SecretStorage.
• Modes - Ask (chat + approval), Plan (research + plan only), Auto (full access).

1.5 Highlights

• No model ships with Silo. It auto-detects whatever you already have pulled in Ollama at startup and lists every one of them in the model picker — nothing hardcoded, nothing to configure.
• Tool-calling and reasoning support are read from Ollama's own per-model capabilities instead of a fixed list of known model names, so any model you pull works correctly right away.

1.4 Highlights

• Resource-safe startup: Silo no longer warms Ollama or auto-loads GGUF models by default, so opening VS Code does not immediately fill RAM/VRAM.
• New Silo: Stop Local Models command stops local llama.cpp/Ollama runtimes when you want memory back.
• UI strings and latency metadata are normalized to UTF-8/ASCII-safe text to avoid broken encoding artifacts in the webview.
• Backend accepts loopback clients only by default and CORS is restricted to VS Code/localhost origins.
• Tool outputs redact common API key, token, password, and bearer-token shapes before returning to the chat loop.
• Command, git, search, directory, and MCP outputs are clipped consistently to keep context small.
• Chat history can be emptied completely; the empty state renders once in the center of the history panel.
• Silo tools can access absolute filesystem paths outside the opened workspace when you provide or request those paths.

Requirements

Speed path

Simple prompts use an instant route: no project tree, no open files, no RAG, no diagnostics, no git diff, no tool schemas, and only a short history window. Code/edit/search prompts still get workspace context when it helps. Local responses can show latency metadata: model, fast/workspace route, first-token time, total time, and estimated prompt size.

Silo can also use local llama.cpp servers for no-tool local turns. Configure silo.llamacpp.serverPath and GGUF model paths once in VS Code settings. By default, models load on demand instead of during VS Code startup. Set silo.startupModelLoading=fast only if you want the Phi server preloaded; set silo.startupModelLoading=heavy plus silo.llamacpp.qwenAutostart=true only if you explicitly want the heavy Qwen/MoE server preloaded. If an accelerator is not configured or fails to start, Silo silently falls back to Ollama. Agentic/tool turns stay on Ollama for compatibility.

Use Silo: Configure llama.cpp / GGUF from the Command Palette to select llama-server.exe and a .gguf model without editing settings manually. Phi-looking files are saved to the fast server; Qwen/DeepSeek/MoE-looking files are saved to the heavy server with heavy autostart disabled to protect RAM/VRAM. Silo also auto-detects Phi, Qwen, and MoE .gguf files in common folders such as C:\ALL\Models\Silo, workspace models/, and ~/.silo/models.

Large MoE GGUFs can use a separate heavy-model profile without changing Phi. Configure silo.llamacpp.qwenExtraArgs, silo.llamacpp.qwenGpuLayers, and silo.llamacpp.qwenMoePreset. The auto preset detects MoE-looking filenames and adds conservative CPU-expert offload, RAM preload, memory lock, and KV-cache settings when the model size fits safely in system RAM. Use speed to tune --n-cpu-moe through silo.llamacpp.qwenMoeCpuLayers; use long-context for a smaller KV cache. This path is useful for Qwen/Kimi/GLM/DeepSeek-style MoE GGUFs exposed as the silo-qwen alias.

Recommended 64 GB RAM coding profile: Qwen_Qwen3.6-35B-A3B-Q6_K_L.gguf, silo.llamacpp.qwenMoePreset=speed, silo.llamacpp.qwenMoeCpuLayers=20, and silo.llamacpp.qwenContextSize=65536. If memory pressure appears, lower to Q5/Q4 or reduce context before changing the model family.

When a Qwen GGUF is configured, the model picker shows the GGUF-derived name, for example Qwen3.6 35B-A3B, instead of the fallback Ollama model label.

DeepSeek V4 can be added as a cloud/OpenAI-compatible provider today. Local DeepSeek V4 Flash is experimental: current community GGUF builds require a custom llama.cpp branch and the native GGUF is far larger than typical 64 GB RAM systems can preload. Silo will run it only if you provide a compatible local server/model path.

Use Silo: Stop Local Models if a local model has filled memory and you want to return RAM/VRAM to the system.

• VS Code 1.90+
• Python 3.10+ (for the local backend)
• Ollama if you want the local model path
• Optional: API keys for cloud providers

Setup

1. Install the extension

Search Silo in the VS Code Extensions panel, or install the .vsix directly.

2. Start the backend

git clone https://github.com/danielmadridg/silo.git
cd silo/backend

python -m venv .venv
# Windows
.venv\Scripts\activate
# macOS / Linux
source .venv/bin/activate

pip install -r requirements.txt
uvicorn main:app --host 127.0.0.1 --port 8942

On Windows you can double-click scripts/start-backend.bat.

3. Pull a local model

Silo doesn't ship with a model of its own — pull whatever you want to use, and Silo picks it up automatically:

ollama pull qwen2.5-coder:7b

Any model already in Ollama shows up in Silo's model picker on its own, with tool-calling and reasoning support detected from Ollama's own capability metadata for that model — nothing to configure. If you have more than one, the auto-model toggle routes complex prompts to your largest and simple ones to your smallest.

Two optional example Modelfiles (backend/Modelfile-qwen, backend/Modelfile-phi) are included if you want tuned sampling params and context budgets for a fast/heavy pair — this is entirely optional and Silo works the same without them. Edit backend/config.py to change the default model.

4. (Optional) Add a cloud model

Open the Silo sidebar -> click the model picker -> + Add AI -> pick OpenAI / Anthropic / Gemini, paste your API key.

Features

Project structure

Silo is organized by responsibility so future changes are easier to locate:

• Extension activation and command registration: extension/src/extension.ts
• llama.cpp/GGUF lifecycle and MoE presets: extension/src/services/llamaCppService.ts
• Chat panel shell and sidebar state: extension/src/panels/SiloChatViewProvider.ts
• Standalone chat tabs: extension/src/panels/controllers/SiloChatPanelController.ts
• Chat persistence, cloud model registry, and IDE signals: extension/src/panels/stores/
• Webview document, styles, models, and browser scripts: extension/src/panels/webview/
• Backend chat routing, request models, stream helpers, and policy: backend/routers/chat*.py
• Backend tool schemas and filtering: backend/tool_registry.py
• Backend tool implementations: backend/tools.py

Modes

In Silo 1.4.1, Ask mode can also prepare file edits as approval previews. The UI shows a unified diff and waits for Apply or Reject. Auto/Edit mode keeps full access and applies changes directly.

Slash commands

Type / in the input to open an autocomplete picker. Arrow keys to navigate, Enter to confirm.

• /clear - start a new chat
• /compact - summarize the current chat to save context
• /review [ref] - AI review of git diff vs ref (default HEAD~1)
• /search <query> - web search via the agent
• /export - save the current chat as Markdown
• /mode ask|plan|auto - switch mode
• /model [id] - switch model (or open picker)
• /help - list all commands

Plan mode

Plan mode is a research-first workflow modeled on Claude Code:

1. Think - model opens a <think> block to reason about scope and ambiguities.
2. Explore - uses read-only tools (read_file, search_content, git_status, web_search) to understand the codebase.
3. Clarify - calls ask_user with a question and 2-4 concrete options. UI shows clickable option buttons + a free-text input.
4. Plan - produces a numbered plan with files to touch, risks, and open questions.
5. Todo - emits a checklist in the side panel.
6. Hand off - ends with a "-> Switch to Auto mode to implement" button.

Auto-model selector

The model-routing button in the toolbar picks between whichever local models you actually have installed, ranked by parameter count:

• Short / conversational / simple questions -> your smallest installed model
• Code, search, analysis, refactor, multi-step tasks -> your largest installed model

Auto-detection uses structural signals instead of language keyword lists: file paths, @ mentions, fenced code, diffs, stack traces, command-shaped lines, and prompt size. Ambiguous short prompts stay on the smaller model.

You can disable Auto and pick a model manually from the model picker.

Thinking toggle

The Thinking switch at the bottom of the model picker controls reasoning mode for models that support it:

• ON - the model generates a <think> block before responding. Streamed live to the Reasoning panel. Best for complex tasks.
• OFF - the model replies immediately without reasoning. Best for fast iteration on simple changes.

Whether a model supports this is read from Ollama's own capabilities for that model — the toggle is hidden or ignored for models that don't report thinking support.

Cloud providers

Bring your own key. Add AI includes compact presets for OpenAI, Anthropic Claude, Google Gemini, DeepSeek, xAI, Groq, Mistral, OpenRouter, Together AI, Fireworks, Perplexity, Cerebras, NVIDIA, Moonshot, Qwen, and any custom OpenAI-compatible endpoint.

Keys are stored in context.secrets (VS Code SecretStorage - OS keychain on Mac/Windows/Linux). Never written to disk as plain text.

Workspace context

Silo includes workspace context only when the prompt needs it. Simple questions stay on the instant route; code/edit/search prompts can include:

• The active file + selected/open tabs
• VS Code diagnostics (problems)
• git diff (unstaged changes)
• SILO.md or CLAUDE.md memory from your workspace root
• BM25 workspace snippets

You can also mention context explicitly:

• @backend/tools.py includes a file
• @backend includes a folder summary
• @codebase asks Silo to lean on workspace retrieval
• @docs tells Silo to use documentation/web tools when available

Persistent project instructions can live in SILO.md, CLAUDE.md, .silo/SILO.md, .silo/CLAUDE.md, or .silo/instructions.md.

Silo's automatic context focuses the current workspace for speed, but file tools also accept absolute paths. If you give Silo a path outside the opened workspace, it can inspect or edit it within the current Windows/VS Code permissions.

Checkpoints, checks, and MCP

Before Auto/Edit changes or approved Ask edits, Silo writes a checkpoint patch to .silo/checkpoints/. After Auto/Edit file changes, Silo detects common project checks and streams the result back into the chat so it can fix failures.

MCP servers can be configured in .silo/mcp.json:

{
  "mcpServers": {
    "example": {
      "command": "node",
      "args": ["path/to/server.js"],
      "env": {}
    }
  }
}

Multi-chat history

The sidebar keeps your previous chats. Empty chats are discarded. Long chats auto-compact.

Inline tools

• Analyze file - full review of the active file
• Refactor selection - right-click -> Silo: Refactor Selection
• Explain selection - right-click -> Silo: Explain Selection
• Inline completions - Tab to accept

Configuration

Security

• Backend binds to 127.0.0.1 by default and rejects non-loopback clients unless SILO_ALLOW_REMOTE=1 is explicitly set.
• CORS is restricted to VS Code webviews and localhost/127.0.0.1 origins.
• API keys live in VS Code SecretStorage (OS keychain).
• Tool outputs redact common API key, token, password, and bearer-token shapes before returning to the chat loop.
• No telemetry. No analytics. No outbound calls except to your configured provider.
• Source on GitHub - audit anything.

Source

github.com/danielmadridg/silo

License

MIT - see LICENSE.