Lean AI

Your codebase already has an architect. It just needs tools.

Lean AI is an agentic coding assistant that reads your project, plans changes, and executes them — all inside your editor. Give it a task in plain English, review the plan, and watch it work.

Run it fully local with Ollama, or connect to OpenAI and Anthropic when you need heavier reasoning. No cloud account required to get started.

Features

• No prompt engineering needed — describe what you want in plain English. A built-in chat assistant helps you refine your idea into a detailed task before the 6-phase planning pipeline takes over.
• Plan first, then execute — a 6-phase planning pipeline reads your codebase, traces data flow, and produces a structured plan. You approve before anything changes. Thinking and content tokens stream live during each phase so you can watch the agent reason in real-time.
• Three workflow modes — /agent for full planning, /fix for quick bug fixes, /request for open-ended research and documentation tasks.
• Architecture review mode — /improve-codebase-architecture reviews the current repo using code, project context, session history, memories, and recorded decisions to suggest high-leverage deepening opportunities.
• Multi-provider — Ollama (free, local), OpenAI, and Anthropic. Switch from the settings panel without restarting.
• Dual-model pipeline — use a fast local model for exploration and implementation, hand off to a cloud model for reasoning-heavy planning phases.
• Built-in code quality — auto-runs your linter and tests after every change, with LLM self-correction on failure.
• Reference library — drop internal docs (PDF, EPUB, Word, Markdown) into .lean_ai/reference/ for context-aware plans.
• MediaWiki integration — connect to an internal wiki so the agent can search and read company documentation while working on tasks. Supports authenticated and public wikis.
• Integrations — two-way sync with GitHub, Jira, and ServiceNow for session summaries, task linking, and optional commit attribution.
• Git-native — every task runs on its own branch. Approve to merge, reject to discard.
• 19 scaffold recipes — bootstrap new projects with /scaffold.
• Notes & TODOs — save notes from chat with /note, and the LLM can save notes and track project TODOs via built-in tools. Notes are auto-categorized by project with tags and searchable from the Notes panel.
• Voice interaction — optional Speech-to-Text, Text-to-Speech, and wake word detection for hands-free coding (see Voice below).
• Vision support — paste or drag-and-drop screenshots and images into the chat. A vision model describes visual content so the LLM can reason about UI mockups, error screenshots, and diagrams (see Vision below).

Quick Start

1. Install the extension

Install from the VS Code Marketplace or OpenVSX. On first activation, the extension automatically creates a Python virtual environment and installs the backend server — no manual setup required.

2. Install Ollama

Download Ollama and pull a model:

ollama pull qwen3-coder:30b

3. Open a project and run /init

Type /init in the chat panel to index your workspace and generate project context. Then describe what you want built.

Manual backend setup (advanced)

If you prefer to manage the backend yourself, set lean-ai.backendDir or lean-ai.pythonPath in settings. The automatic installer is skipped when either setting is explicitly configured. See the GitHub repository for details.

Slash Commands

Vision

Attach screenshots, UI mockups, error messages, or any image to the chat. A separate Ollama vision-language model describes the image so the main LLM can understand visual content without native vision support.

Setup:

1. Pull a vision model: ollama pull qwen3-vl:8b
2. Set lean-ai.visionModel to qwen3-vl:8b in the extension settings (or LEAN_AI_VISION_MODEL env var).

Usage: Paste an image with Ctrl+V, drag-and-drop a file onto the chat input, or use the attachment button. Images work in both chat conversations and agent workflows — the vision model processes each image in parallel and injects the description into the LLM context.

Voice

Optional voice interaction for hands-free coding: speak your requests, hear responses read aloud, and trigger recording with a wake word.

Setup:

# Install voice dependencies (requires portaudio system library)
# Ubuntu/Debian:
sudo apt install portaudio19-dev
# macOS:
brew install portaudio

# Install Python voice extras
pip install "lean-ai[voice]"

Enable the features you want in the extension settings or via environment variables:

When voice dependencies are missing but settings are enabled, the extension offers to install them automatically.

Speech-to-Text (STT)

Click the mic button in the chat input to record, click again to stop. The audio is transcribed locally using faster-whisper (a CTranslate2-based Whisper implementation) — no audio leaves your machine.

Text-to-Speech (TTS)

Toggle TTS in the chat voice controls. The LLM's responses are read aloud using kokoro-onnx with 58 voices at 24kHz. Sentences stream as they arrive, so speech starts before the full response is generated. Code blocks are automatically stripped so the engine doesn't read code aloud.

TTS model files (~169MB for fp16) are downloaded automatically on first use.

Wake Word

Enable wake word detection for hands-free activation. When the wake word is detected, STT recording starts automatically and the transcribed text is submitted to the chat.

The wake word listener runs on a background thread at 16kHz, using openWakeWord. The default trigger phrase is "Hey Jarvis". While STT is recording, the wake word listener pauses to avoid mic contention, then resumes when recording stops.

Set lean-ai.wakeWordAutoSubmit to true to automatically send the transcribed message after wake word activation (otherwise the text is placed in the input for you to review).

All voice processing runs on CPU only — GPU is reserved for the LLM.

Configuration

Open the settings panel (gear icon in the chat header) to configure:

• LLM Provider — Ollama, OpenAI, or Anthropic
• Model selection — primary, expert, and request models with independent sampling parameters (temperature, top-p, top-k, repeat penalty, context window, max tokens) and thinking mode per model
• Post-validation — lint, test, and format commands
• Search provider — DuckDuckGo, SearXNG, Google, or Bing
• MediaWiki — connect to an internal wiki instance (URL, API path, optional authentication)
• Integrations — GitHub, Jira Cloud, and ServiceNow for two-way task sync and optional Lean AI co-author trailers
• Vision model — Ollama vision-language model for image understanding
• Voice — STT, TTS, and wake word settings

Settings are saved to backend/config.yaml. API keys and integration tokens are stored securely in your OS keychain — never written to config files. For standalone backend usage, secrets can be encrypted in the YAML file.

Requirements

• Python 3.10+ (for the backend server, installed automatically)
• Ollama with a capable model (e.g., qwen3-coder:30b) — or an OpenAI/Anthropic API key
• For voice: portaudio system library + pip install "lean-ai[voice]" (optional)
• For vision: an Ollama vision model like qwen3-vl:8b (optional)

Links

• GitHub Repository — source code, backend setup, and full documentation
• Changelog — release history and recent changes
• Configuration Guide — all environment variables and settings

License

MIT