Swarm Lab

Chat with your project and work with a team of AI agents inside VS Code—with visible steps, reviewable edits, and command approvals.

Start with one assistant. When you need more help, use a lead with sub-agents, a swarm with a shared board, or a fixed workflow such as plan → fix → test → review. Connect a hosted model, a local model, or VS Code's language models. An offline mock lets you explore without a key.

New here? Follow sections 1–4. You do not need to configure a server, agents, workflows, or any advanced feature to start.

Contents

• 1. Install and open
• 2. Connect a model
• 3. Your first conversation
• 4. Your first run
• 5. Find your way around
• 6. Approvals, checkpoints, and sandbox limits
• 7. Code search (RAG)
• 8. Decision models
• 9. Automatic model choice
• 10. Agents, workflows, and extra tools
• 11. Tests, verification, and checked "done"
• 12. Code map and step map
• 13. Memory, history, and project files
• 14. Costs, context, and limits
• 15. Files, pictures, web, and saved results
• 16. Compare setups with Bench
• 17. Optional background server
• 18. Troubleshooting

1. Install and open

What you need

• VS Code 1.95 or later, or a compatible editor supporting that extension API.
• A project folder to open. A small, non-sensitive practice project is ideal for your first session.
• A model connection for real AI answers. Hosted APIs may charge separately; local models require a running model server. The offline mock needs neither.
• Git with at least one commit if you want run-branch safety or repository-based server/Bench jobs. Git is not required for basic chat.
• Your project's own dependencies and test tools if you want the assistant to run its tests.

You do not need Node.js or the source code just to install and use the extension. Those are needed for development or a standalone server.

Install the extension

Open Extensions (Ctrl+Shift+X, macOS Cmd+Shift+X), search Swarm Lab, check that the publisher is joe-el-khoury, and install it.

• VS Code Marketplace
• Open VSX for editors configured to use that registry.

Have a VSIX instead? Extensions → ⋯ → Install from VSIX… → choose the downloaded package. You can also use the VS Code command-line launcher:

code --install-extension swarm-lab-0.36.5.vsix

Then:

1. File → Open Folder… → choose your project.
2. Open the Command Palette (Ctrl+Shift+P, macOS Cmd+Shift+P).
3. Run Swarm Lab: Open.
4. If an update does not appear, run Developer: Reload Window.

2. Connect a model

Open Setup. Choose a provider, specify models where required, and finish with Save settings, Test connection, then Test tool calling.

Keys and model names

1. Click Set API key, select the correct provider, and paste the key. Keys go into VS Code Secret Storage, not project settings.
2. Set a Lead model: used for the assistant, single-agent runs, and leads.
3. Set a Worker model: used for workers, sub-agents, and workflow steps. Using the same model for both is fine to start.
4. Use exact model IDs offered by your provider. With OpenRouter, Pick OpenRouter models helps choose tool-capable models and shows prices.
5. Save and run the connection/tool tests. Open Check → Check everything if anything fails.

Local model: start Ollama or LM Studio first, download/load a model, and enable its API server. Use the exact model ID from that server. For example, inspect the server's /v1/models endpoint. Installing Swarm Lab does not install a local model for you.

No native tool calling? In Setup, try Tool calling → Text. The model writes structured tool requests and the usual approvals still apply. Off is plain conversation without tools; native tool calling is preferred when supported.

Company network: Setup has a network proxy field. For gateway authentication, the key header is normally authorization; some gateways require api-key. Keep secrets out of extra headers/settings files. Do not disable TLS verification to work around certificate problems; configure your company's trusted certificate instead.

3. Your first conversation

Start in Chat with a read-only question:

Explain what this project does. Read the relevant files first, name the files you used, and do not change anything.

Then try a small, specific task:

Find one missing edge-case test for the date parser. Propose the test, explain it, and ask before running it.

• Enter sends; Shift+Enter adds a line.
• The assistant can search and read your workspace. Its activity shows which tools it uses.
• Waiting for you holds proposed edits, commands, questions, and run requests. Review the diff or command before approving.
• Apply accepts an edit; Reject declines it. You can edit a proposal before applying it.
• Stop interrupts the work and cancels pending approval requests. It does not undo changes already applied or external command effects.
• For code-specific help, select code and use the editor's Swarm Lab context menu: explain, fix, write tests, or ask a question.
• Use History to reopen conversations and New chat to start a fresh one.

Good requests include: the goal, relevant files, expected behavior, things not to change, and how to test success. Review generated code and run tests before merging it.

4. Your first run

A chat is an ongoing conversation. A run gives a bounded task to a chosen organization of agents and produces a result and usage report.

1. Save your files. Ideally commit your existing work first.
2. Open Run, write a small task, and choose Single agent.
3. Check limits and safety settings, then click Run task.
4. Follow the activity and answer approvals.
5. If Git safety is active, review the final Keep / Revert all choice. Keeping leaves you on the run branch; review and merge it yourself.

You can also request these from Chat:

/single Explain the test setup without changing files
/tree Investigate input validation in the API and UI; report findings only
/swarm Review error handling across the project; propose a plan before edits
/graph Fix the failing date-parser test using the saved workflow
/ask reviewer What risks did you find in the last run?
/review --working

Configure and save a workflow before using graph mode. Follow-up questions can reach agents still retained after a run; retired tree sub-agents are not available.

5. Find your way around

The live steps and agent activity show model calls, tool calls, edits, and results. Code maps use VS Code's language support, so install the relevant language extension for your project.

6. Approvals, checkpoints, and sandbox limits

Approvals and permissions

For the Swarm Lab engine, file changes are proposed as diffs. Commands ask by default. In Setup you can make edits read-only, disable commands, or let approved command patterns run through an allow-list. Named agents can be limited to particular tools and edit paths.

An allow-list is a convenience, not a security boundary: even an ordinary test script can run arbitrary project code. Review scripts before allowing them. Tools such as MCP and Claude Code have their own permission paths; do not assume each action they take appears as an individual Swarm Lab edit approval.

Commands that cannot be undone (deleting files recursively, force push, git reset --hard, DROP/TRUNCATE, deploys, publishes, cloud deletes, and similar) always ask, even when allow-listed, with a ⚠ cannot be undone warning.

What checkpointing actually covers

If your local working tree is dirty, Git safety asks to stash it or continue without protection. Reverting a run restores the stash; keeping the run leaves earlier work in the stash for you to recover. Run follow-up questions do not retain the completed run's per-turn checkpoints. Git rollback cannot undo network requests, package installs outside tracked files, database changes, or other external effects.

Is it sandboxed?

Local chat and local runs operate in your open workspace, not an isolated copy. A run branch separates Git history, not your operating system.

Server repository jobs and Bench tasks use separate Git worktrees. The server workspace adapter checks file paths and routes edits/commands through policy and approvals. However, approved commands execute in a normal host shell, with the job directory as the working directory and inherited process environment. They are not confined by a per-job container or VM and may access host files, credentials, and the network.

Treat this as approval-based control and Git-based recovery—not a hardened security sandbox. Use a dedicated, least-privileged environment for untrusted projects. Only connect MCP servers and services you trust.

Claude Code engine

Claude Code is optional. Install and authenticate the Claude Code CLI separately, set its executable path in Setup if necessary, and use Test Claude Code before selecting it as an agent/chat engine. Its tool permissions, billing, and confirmation behavior are separate from the ordinary Swarm Lab model loop. Keep its permissions conservative while learning.

7. Code search (RAG)

Agents get a search_code tool that finds the code relevant to a question, even when the code uses other words ("where are expired tokens handled?").

Setup → Code search (RAG):

1. Code search on (default).
2. Embedding model (optional, for search by meaning): for example text-embedding-3-small, nomic-embed-text, bge-m3. Empty = keyword search only, which works with any provider.
3. Embeddings address: an OpenAI-compatible /embeddings endpoint, for example http://localhost:11434/v1. Empty = the chat provider's address (OpenAI-compatible or OpenRouter). Anthropic has no embeddings endpoint: set an address, or stay keyword-only.
4. Chat: add relevant code to each message: the most relevant chunks are added to each chat message. Older retrieved code is removed from the history, since it can be found again.
5. Results per search, Chunks added per chat message, Lines per chunk, Only these files / Also skip (globs).
6. Build index now, then try the Search box.

How it works: files are cut into chunks at functions, classes and headings, and stored in .swarmlab/index/index.json. Search combines keywords (BM25, with code names split: parseDate → parse date) and meaning (embeddings), merged into one ranking. Results give the file, the line range and the code; the agent then reads the exact lines. The index updates by itself after you save, create, rename or delete files: only changed files are re-indexed.

Keyword-only search is free and needs no extra service. Meaning-based search is more capable but requires an embedding endpoint and may cost money.

8. Decision models

Some steps need a decision, not a written answer: which way a workflow goes, which kind of run fits, whether a message even needs code. A decision model such as Jev (TypeSafe AI's "System One" model) reads the situation and returns a typed answer with its probability: a choice among options, a score on a scale, or a yes/no. It writes no text, so it is fast and costs a small fraction of a lead-model call.

It is optional and off by default. When it is on and it is unsure (below Minimum confidence) or fails, Swarm Lab does exactly what it does without it. It never approves an edit or a command.

Set it up

1. Setup → Decision model → Use a decision model.
2. Where:
   • OpenRouter: uses your OpenRouter key (OpenRouter serves Jev through its System One API). Model jev-latest or jev-1.13.
   • TypeSafe directly: Set decision model key → TypeSafe key.
   • Another System One server: a LiteLLM proxy, a company gateway, or another host of the same API. Base URL is the part before /v1/systemone.
   • No decision model: the worker model answers the same typed questions as JSON. Useful to compare, or where only chat models are allowed.
3. Save settings → Test decision model: it asks about a sample support ticket and shows the answers, their confidence, the time and the cost.

Where it is used

Each one has its own switch:

Every decision appears in the step map (◆), the activity view and the log, with its answer, confidence, whether it was used, and its cost.

Decision steps in a workflow

In the Workflow tab, Add a decision step. It has a question, what it reads (default {last}, the previous step's output), and its answers:

• Choice: options as name: what it means; each option routes to a step.
• Yes or no: routes yes and no; Yes when at least sets the threshold.
• Score: levels described in words, lowest first; each level number routes to a step.
• else catches the case where there is no good answer.

A decision step works even with the decision model off: the worker model answers it. With the decision model on, it costs no lead- or worker-model call.

9. Automatic model choice

For each task, Swarm Lab can pick the cheapest model from your list that is good enough, instead of always the lead model.

1. Setup → Automatic model choice → Strategy:
   • Cheapest: small models unless the task is hard (then medium).
   • Balanced: small for simple tasks, medium for ordinary ones, large for hard ones.
   • Best: large models unless the task is simple (then medium).
2. The model list: each model with a size class (small, medium, large). Add the lead and worker, add others, and Size classes by price sorts them for you. Uncheck Tools for a model that cannot call tools; give its Context when you know it. With an empty list, the worker model is small and the lead model is large.
3. How hard a task is: by rules (free: words like refactor, debug, design; the length; the amount of code), or by the decision model (Setup → Decision model).

Then the choice is narrowed: agents that use tools only get models that can; a conversation longer than a model's context skips it; when the next call would pass the cost limit, a smaller model that fits is used. Move to a bigger model after a failure (on by default) retries a failed call on the next class up.

Where it applies: the chat (its Model picker says Default (Setup), Auto · cheapest / balanced / best, or one model), runs (the Setup strategy), and benches (setups with "leadModel": "auto:cheapest"). A model you set for an agent or a workflow step is never changed; Claude Code agents keep theirs.

Each choice appears in the step map (◆ auto model) and under the answer, with the reason for each choice.

10. Agents, workflows, and extra tools

Make a specialist

In Agents → Add an agent, give it a name, description, and instructions. Choose its model/engine, tools, and May edit paths. Start with a read-only reviewer; choose none for edit permission. Save permissions, then select it in Chat's Talk to picker, Run's Run as control, or @name in a message.

A useful starter team is a read-only planner, a fixer limited to the source/test folders, and a checker allowed to read files and run tests. More agents are not automatically better or cheaper.

Prompts

In Prompts you can inspect and override the instructions for each role (single, tree-lead, tree-worker, swarm-lead, swarm-worker, graph-node, chat) and each named agent, with a Preview the final prompt that shows exactly what an agent would receive.

Use a workflow

In Workflow, choose Start from a template → Use this template. Review the steps and especially the test command, then Save workflow. Start with Bug fix, Safe refactor, Write tests, Write docs, Code review, or Dependency upgrade. Configure the project's dependencies before running test steps.

Workflow steps are agent steps (an agent, a task, and where to go next) or command steps (such as npm test, with If it passes / If it fails routes). Tasks can use placeholders like {task}, {last}, and {node:<step>}. A step can loop with Stop looping after N visits.

Project-management workflows can produce tickets, sprint plans, release notes, status reports, and risk registers. Optional judge-calibration workflows measure an LLM judge; they are advanced evaluation tools, not required for coding.

Add rules, skills, or MCP

• Rules provide project-wide instructions. Use .swarmlab/rules.md or topic files under .swarmlab/rules/.
• Skills are reusable procedures managed in Tools. Each has a name, a one-line when to use it, and instructions. Review them before adopting; agents load a skill with use_skill when a task matches. You can also ask the assistant to "make a skill of what we just did."
• MCP servers expose external tools such as issue trackers or databases. Add trusted local/remote servers in Tools, review authentication and permissions, and test the connection. External tools may have irreversible side effects.

11. Tests, verification, and checked "done"

run_tests

Agents get a run_tests tool (on by default). It runs your test command—all tests, some files or folders, or only tests whose name matches—and returns a short report instead of the whole output. It reads pytest, Jest, Vitest, Mocha, node --test, go test, cargo test and dotnet test; for other runners it keeps the exit code and the last 40 lines. Paths and test names are checked character by character, so nothing can be chained or redirected. It goes through the same approval as any command.

"Done" is checked, not trusted

An agent saying it is done is a claim. When an answer or a run changed files, Swarm Lab checks it independently:

• complete edits: no placeholders left instead of code and no merge-conflict markers in the changed files;
• tests: if the agent ran the tests after its last change and they passed, that counts; otherwise the check runs them (it asks you unless the test command is on the allow-list).

The answer shows ✓ verified (tests passed after the last change), ✕ check failed, or ◌ not verified (no test could be run—it never says verified without evidence). When the check fails, the agent gets one more turn with what failed.

12. Code map and step map

Step map

Every run and chat message shows what happened, step by step. In Run, switch the map to Steps; in Chat, click Steps. Marks include a circle for a model call (hover for tokens), a square for a tool, a blue ring for code search, a green/red square for an edit applied/rejected, a $ box for a command, and a ◆ diamond for a decision. Click a mark with a file to open that file at that line.

In VS Code itself, during a run or chat message: Explorer badges on the files the agents touched (R read, E edited, P proposed), line highlights on the lines an agent read, and the Swarm Lab: agent activity view with one entry per agent and step.

Code map

What calls what, drawn from your editor's own language support (the same data as VS Code's Call Hierarchy). Put the cursor on a function, right-click → Swarm Lab → Show call map, or open the Code map tab and click Build map. Each function is a box colored by its file; callers on the left, callees on the right. Click a box for its callers/callees; double-click to open the code; Open as Mermaid exports the map as a flowchart.

Agents also get two read-only tools, code_symbols (a file's classes, functions and line numbers) and call_graph (what a function calls, or what calls it), so they follow the code for a few hundred tokens instead of reading whole files. These tools are not offered on the server, which has no language servers.

⚠️ Dynamic calls (callbacks, obj[name](), dependency injection) are often missing from any call hierarchy, so treat the map as a guide, not proof.

13. Memory, history, and project files

Memory

Memory has three scopes, stored as readable Markdown in .swarmlab/memory/ (open, edit and commit it like any file):

After each answer, "Remember this?" suggests up to three things worth keeping (a decision, a preference, a fact that was hard to find) and new topics. Each line can be edited, ticked or not, and sent where you choose. Nothing is saved or used before you keep it. The Memory panel (chat → Memory) lists items with where they came from, how far to trust them, and how often they were used, with Edit, Wrong, Approve, and Delete.

Every item records its source. What an agent saves right after reading a web page or a connected (MCP) server is marked from an outside source and never goes to the project memory without you: memory that can be taught can be poisoned.

History and project files

History restores conversations. Memory stores reusable facts, not just transcript text. New chat starts a new conversation; it does not necessarily clear durable memory. Saved chat state is not a resumable checkpoint for every in-progress run.

Swarm Lab keeps readable configuration and working data in .swarmlab under the project:

API keys saved through the extension use Secret Storage. Chat histories, indexes, memory, and generated reports can still contain sensitive information. Git safety intentionally excludes .swarmlab from its commits/reverts; add .swarmlab/ to .gitignore if you do not want it in the repository. Review what you back up, share, or commit.

Migrating from Cline, Roo Code, or Continue? Run Swarm Lab: Import settings from Cline, Roo Code or Continue. Review the preview before importing rules, agents/modes, prompts, MCP, and compatible settings. API keys are not copied; add them separately.

14. Costs, context, and limits

The extension is free; your model provider may charge for calls. Multiple agents and workflow retries can multiply usage.

• Begin with Single, a small task, a modest token/step budget, and limited parallel calls.
• Set per-message and daily cost limits in the chat's Costs controls; configure run budgets and worker/step limits in Setup.
• Check the model's prices. OpenRouter pricing can be loaded automatically; other providers may need prices entered in Setup. Cost unknown is not free: dollar-based limits cannot be treated as reliable without correct pricing.
• The usage details under answers show calls, tokens, models, and known cost. Provider billing remains authoritative.
• Automatic model choice and optional decision models can reduce cost, but are not required to get started.

Open Context in Chat to see what fills the model's context (instructions, tools, history, your message, attached files, code found, and memory, in tokens), summarize old messages, remove old tool outputs/attachments, and adjust input/output limits. Summaries are not verbatim archives; the saved conversation history is separate.

If an answer stops at a step limit, review its progress report and choose to continue if appropriate. Raise limits gradually rather than removing them. Step mode can pause before model calls or tools while you learn. Model calls at once (Run tab, Setup → Limits) caps parallel model calls for the whole window: use it for provider rate limits or a local model that serves one request at a time.

15. Files, pictures, web, and saved results

• Attach context: click ＋ Files or type @ to choose workspace files. Attachments can include CSV/Excel, Word, PDF, PowerPoint, notebooks, ZIP entries, text, and pictures. Large documents are read in parts; image understanding requires a suitable model.
• External data: use Run → Data to add input files/folders and choose an output folder. Agents read with list_data and read_data (which reads each kind the way a person would). Give access only to the data needed for the task.
• Results: ask for a saved report, table, JSON, or figure. Data outputs default to the project's .swarmlab/out/data folder unless configured otherwise. Code edits still target permitted workspace paths; output-folder settings do not redirect every possible write or shell command.
• Pictures: annotate or crop a picture before sending it (✎ opens the editor: pen, highlight, arrow, box, text, crop). Answers can contain tables, highlighted code, and Mermaid diagrams; diagram export supports PNG, PDF, and SVG.
• Reuse an answer: edit an answer/code block/diagram (Edit on hover), or ask to save a numbered code block to a named project path. Saving into the workspace uses the approval flow.
• Web research: enable the chat's 🌐 Web control when you want web search/page reading. Search may require configuration and a separate search-provider key. Built-in web access restricts private-network destinations and treats page content as untrusted; it does not apply the same restrictions to arbitrary shell commands or external tools.

Privacy: files, attachments, retrieved code, and tool results included in a prompt are sent to the selected model service. Embedding/search providers and external tools may receive their own inputs. A local model avoids sending model prompts to a hosted API only if your full setup—including embeddings and integrations—is local. Do not provide secrets or confidential data without checking your organization's policy.

16. Compare setups with Bench

Bench gives the same tasks to multiple model/agent setups and compares pass rate, tokens, cost, and time.

1. Open Bench (or Swarm Lab: Open Bench in the Command Palette).
2. Choose New starter suite, or save a real task with Save as bench case.
3. Edit the tasks, setups, and checks. Keep the first suite small; each task × setup × repeat is another run.
4. Use a Git repository with an initial commit. Worktrees start from a committed base, not unsaved/uncommitted editor changes.
5. Run it, inspect failures, and optionally Set as baseline for future comparisons.

Checks include tests (the test command passes), command, files (changed/unchanged globs), answer (contains/matches), and judge (a rubric scored by the decision model or the worker model). A run passes when all its checks pass. Setups can compare modes (single, tree, swarm, graph), models (including "leadModel": "auto:cheapest"), the decision model ("decisions": true), and skills.

Bench is unattended: allowed edits occur inside task worktrees and commands follow its allow-list/refusal policy rather than waiting indefinitely for clicks. Tests can still execute host code, so use a trusted project and isolated environment where needed. Judged scores are not a replacement for objective tests.

17. Optional background server

Skip this section if you only want interactive work in VS Code. The server runs jobs outside the chat session, with an API, browser dashboard, approvals, and worktrees for repository jobs.

Easiest route: the Server tab

Choose and save an API provider/model in Setup, save its key, then use Server → Start local server. The launcher synchronizes supported provider/model settings and forwards the saved provider key to the child process without writing it into the configuration. Restart after changing those settings. VS Code language models are not available to a standalone server; use an API provider, local endpoint, or mock.

For an existing server, enter its address in Server, set its server access key, save/test the connection, and choose the default repository. The server access key is separate from a model-provider key. Submit a small single-agent job and answer approvals in Server.

Standalone route: from the source folder

Install a current Node.js LTS release and Git, then:

npm ci
npm run bundle
node dist/server/main.js init swarmlab.server.json
node dist/server/main.js hash-key

Edit the generated configuration before starting:

• Set the provider type, endpoint, and exact lead/worker model IDs. The generated example uses a local OpenAI-compatible endpoint with placeholder model names.
• Provide the model key through the environment variable named by provider.apiKeyEnv, if required. A standalone process cannot read VS Code Secret Storage.
• Set repos.app.path to a real Git repository and baseRef to an existing branch/commit. A source ZIP without Git history is not a valid repository base.
• Replace the example apiKeys entry with a key's SHA-256 from hash-key, using the sha256 field and appropriate roles; keep the original key privately for the client. Alternatively supply the access key through the configured keyEnv environment variable.
• Keep the host on 127.0.0.1 for local learning. Review approval policies, limits, and data directory permissions.

Then validate and start:

node dist/server/main.js check --config swarmlab.server.json
node dist/server/main.js serve --config swarmlab.server.json

Open http://127.0.0.1:8787/ and enter your server access key. Connect the extension to that same address/key. A chat request can submit a remote job after connection:

/remote single Explain the configured repository without changing files
/remote single --no-repo Draft a short project kickoff checklist

Do not expose an unauthenticated server. Before remote use, configure access keys, TLS/reverse-proxy protection, network restrictions, and a dedicated runtime account. Repository worktrees are not a host security boundary. PostgreSQL, separate workers, profiles, webhooks, and experiments are advanced options; start with the local file-backed server.

18. Troubleshooting

Start diagnostics with Check → Check everything and View → Output → Swarm Lab. When reporting an issue, include the extension version, editor/OS, provider type, sanitized error, and reproduction steps—never API keys or sensitive prompts. Report an issue. Read the author's articles: @jelkhoury880 on Medium.

License: MIT. Bundled third-party components retain their respective licenses, including Mermaid (MIT), highlight.js (BSD-3-Clause), and undici (MIT).