LlamaNexus for VS Code

Chat with, explain, and edit code using your local LlamaNexus proxy (your Go proxy in front of llama.cpp), directly inside VS Code — no Open WebUI needed.

It talks to LlamaNexus's OpenAI-compatible endpoints:

• POST {baseUrl}/openai/v1/chat/completions (streaming SSE, with tool-calling support)
• GET {baseUrl}/openai/v1/models (includes each model's real context size and whether it's currently loaded)
• GET {baseUrl}/status (connection check, plus current RPC worker mode/roster)
• POST {baseUrl}/api/set-ctx-size (change a model's ctx-size and restart to apply it)
• POST {baseUrl}/api/eject-model (unload a model from the server)
• POST {baseUrl}/api/remove-model (permanently delete a downloaded model)
• POST {baseUrl}/api/rpc/mode, /api/rpc/workers, /api/rpc/workers/remove (toggle single-node/RPC mode and add/remove RPC workers)
• POST {baseUrl}/ollama/api/pull (download a not-yet-present model, streaming progress)

Features

Sessions

Click the LlamaNexus icon in the Activity Bar for a Sessions sidebar: "+ New session", a live-filtering "Search sessions..." box, and a list of every session with rename (pencil) and delete (trash bin) controls. Sessions are per-workspace and persist (including full history, response stats, and any reasoning/thinking text) across window reloads — reopening one replays its prior conversation exactly as it looked before. Selecting or creating a session opens its chat in its own editor tab.

Deleting a session moves it to a collapsible Trash section (collapsed by default) at the bottom of the Sessions view instead of erasing it immediately - restore it or permanently delete it from there, or use the "Empty trash" icon on the Trash section's title row to permanently delete everything in it at once. Move-to-trash, permanent delete, and empty-trash all confirm by default; llamanexus.autoAcceptTrash skips the confirmation for the move-to-trash step specifically (permanent delete and empty-trash always confirm).

Chat

LlamaNexus: Open Chat (Ctrl+Alt+L / Cmd+Alt+L), or click a session in the Sessions sidebar — each session is its own editor-tab chat with:

• Streaming responses, labeled with whichever model actually answered.
• While waiting on a cold model load or a queued request, the reply's role title shows a rotating placeholder word (Thinking, Chewing, Pondering, ...) instead of the model's name — each word fades in character-by-character, pulses gently once fully typed, then crossfades into the next word (fading out while the next one reveals underneath, no gap in between) every 5 seconds. The real model name (and that message's timestamp) only appears once generation actually starts. That same rotating word then keeps showing below the reply for as long as the turn is still in flight, so a pause after the model has already started streaming text (e.g. right before it emits a tool call) still shows something happening instead of going quiet.
• Thinking/reasoning models (anything llama-server streams reasoning_content for) get their reasoning shown in a collapsible "Thinking..." section, separate from the final answer and not resent to the model on later turns.
• Markdown-ish rendering: #/## headers, **bold**, *italic*/_italic_, `inline code`, and -/* lists, plus full fenced code blocks.
• Icon-only Copy, Insert, and Save buttons on every code block (each briefly shows a checkmark to confirm). Insert drops the code at your cursor in the last editor you had focus in (works even though focus is on the chat tab when you click it); Save opens a native save dialog, defaulting to a filename guessed from the fence's language tag (e.g. python → snippet.py).
• User messages are shown in their own card, with a copy button next to the timestamp. read_file/write_file/patch_file/run_bash calls each get their own collapsible titled note (Read/Write/Patch/Bash, bold and white, with the file name shown after the title for file tools) with the relevant content underneath; other tool calls (e.g. list_files) show as a generic collapsible system note. All collapsed by default — click to expand.
• Attach one or more files to a message (the + button) — each shows as a removable chip while composing and as a read-only chip on the sent message, both live and after reopening the session. Attached content is folded into the message sent to the model (200 KB limit per file; binary files are rejected).
• A context-usage bar (used vs. total tokens) that fills as the conversation grows, sized from the model's real ctx-size (see Model tools below) or your manual llamanexus.contextSize setting as a fallback. If the server reports the model's trained context length (GGUF n_ctx_train; requires LlamaNexus 0.1.7+), it's shown in parentheses after the total, e.g. 512 / 8192 (32768).
• Automatic context compaction: once usage reaches llamanexus.autoCompactThreshold% of the context size (default 85, set to 0 to disable), the older part of the conversation is summarized into one message before the next is sent, keeping the most recent llamanexus.autoCompactKeepMessages messages (default 6) verbatim — the cut point always snaps back to the start of a full user turn, so a tool-call chain is never split across the boundary. Checked both before you send a message and between each hop of a chained tool-calling turn (e.g. working through an approved plan), so a long unattended run compacts along the way instead of only ever being checked once you send something new. The summarized-away messages are replaced in the transcript by a single collapsible "Compacted" note showing the summary; trigger it manually any time with /compact or by clicking the ring described below.
• A red ring next to the / command button gives early warning: it appears once usage reaches half of the auto-compact limit and fills in as it gets closer, and clicking it compacts immediately.
• Response stats (prompt/response/total tokens, elapsed time, tokens/sec) under each assistant reply.
• Every message's role title (You, the model name, System, Error) shows a timestamp on the right, formatted per llamanexus.timestampFormat and persisted with the session. For assistant replies it's captured when generation actually starts (not when the placeholder message appears), so it reflects the real response time.
• Message input history: press Up/Down (with the cursor at the start/end of the box) to recall previously submitted text — chat messages and /ctx-size//worker commands alike — with your in-progress draft preserved and restored once you navigate past the newest entry. Seeded from the session's history, so it survives reopening.
• Scrolling up to reread earlier messages sticks — streaming tokens, tool/system notes, and plan updates only auto-scroll to the newest message if you were already near the bottom. Sending your own message always scrolls down regardless, and reopening a session still lands on the newest message.
• A Plan / Edit mode toggle next to the model picker. In Plan Mode the model can only read/explore (read_file, list_files, update_plan) — write_file, patch_file, and run_bash are rejected outright, before their confirmation dialog would even appear, whether attempted as a real tool call or as hallucinated literal text — so it can safely research and propose a plan without being able to act on it. The button turns visually distinct in Plan Mode, the composer placeholder changes, and a system note announces every switch in the transcript. Persists per session and is restored when reopened; defaults to Edit Mode.
• A Plan panel above the composer: when the model calls update_plan to lay out a multi-step or multi-file task, its steps show as a collapsible checklist that updates in place as status changes, rather than as new chat messages. The first proposed plan asks for confirmation the same way a file write does, with a third "Request changes..." option (see Confirmations below) alongside Allow/Deny for sending the model feedback instead; once approved, the model works through the remaining steps without pausing to ask whether to continue, and - if the session was in Plan Mode - it's switched to Edit Mode automatically (with a system note explaining why), since none of those steps could otherwise proceed. Once every step is completed, the next plan asks for approval again rather than inheriting the finished one's approval. Persists with the session and survives being collapsed/expanded; a dismiss button clears it.
  • If a later update_plan call changes step content (not just status) after the plan was already approved, a system note flags that the model revised the plan mid-task, since a mid-flight reshape proceeds without a fresh confirmation click.
  • If a turn that did tool work ends while the plan still has unfinished steps, a system note reports how many steps aren't marked completed — checked structurally against the plan's own state, not by guessing whether the model's closing summary "sounds done", so an overconfident summary can't silently contradict it.
  • Models reliably lay out a plan up front but often just power through the real work afterward without calling update_plan again on their own — if the plan still has unfinished steps and the model makes 3 file-write/patch/bash calls in a row without touching it, the next one's result gets a nudge back toward update_plan.

Controls sidebar

A collapsible sidebar on the right edge of the chat (click the sliders icon strip — labeled "Controls", rotated vertically while collapsed — to expand) holds two independently-collapsible sections, so opening one to make a change never dumps the other open too.

Tools — each session can independently enable/disable file tools, the bash tool, the planning tool, and agent delegation, defaulting to the corresponding global llamanexus.enable* setting when left alone. Each row has two checkboxes: the effective value on the left (click to override; toggling it back to match the global value clears the override again) and a read-only, greyed-out mirror of the current global setting on the right, so the global state stays visible without opening Settings.

Sampling — mirrors Open WebUI's per-chat "Advanced Params": Temperature, Top K, Top P, Min P, Frequency Penalty, Presence Penalty, Repeat Penalty, Repeat Last N, Mirostat (+ Tau/Eta), Seed, Max Tokens, and Stop Sequences. Each field has its own enable checkbox — unchecked fields are left out of the request entirely (the server/llamanexus.temperature/llamanexus.maxTokens defaults apply); checking one and setting a value makes it stick for every message from then on. Enabled values persist with the session, independent of the global settings, and are restored the next time that session is reopened. Unchecked fields display each setting's real default (e.g. Temperature 0.8, Top K 40) rather than blank or a slider's minimum, so you can see what the model would actually use before deciding to override it — unchecking a field immediately reverts its display back to that default too.

Server discovery

A row under the context bar lists every LlamaNexus serve instance found on your LAN (requires LlamaNexus 0.1.5+, which broadcasts a UDP heartbeat on llamanexus.discoveryPort, default 50050). Each session picks its own server independently — click a row to select it for the current session; the choice persists with the session and is restored (if that server's still online) the next time it's reopened. Until a server is selected, the composer is disabled with a prompt to choose one.

The selected server's row expands (click it) to show its RPC worker mode/roster, when running in --rpc/--discovery mode:

• A button to toggle between single-node mode and RPC mode (remembers whichever RPC source — discovery or a manual roster — was last active).
• Per-worker remove (manual mode only; the roster is read-only while auto-discovery is active).
• The /worker <ip:port> (or explicit /worker add <ip:port>) slash command to add a worker, and /worker rm <ip:port> to remove one.

Each row also has a 🤖 icon, independent of the main row-click selection: click it to mark that server as this session's agent server for run_agent (see Model tools below) — typically a separate single-node instance with a small/fast model, since the main server above may already be fully committed to one (possibly RPC-distributed) model. Persists with the session; click the lit icon again to unset it.

Model tools (function calling)

When the connected model/server supports tool calling, LlamaNexus can:

• read_file — read a file in your open workspace.
• patch_file — make a targeted edit by replacing one exact snippet of a file's current text, without rewriting the whole file. Preferred over write_file for existing files, since it avoids re-emitting entire files as one giant argument (which risks hitting the token cap on large files). Always asks for confirmation (see below) before touching disk, showing a diff preview of the change.
• write_file — create a new file or fully rewrite one. Always asks for confirmation (see below) before touching disk, showing what would change.
• list_files — list a directory's contents.
• run_bash — run a bash command directly in your workspace root, without writing it to a file first. Off by default (llamanexus.enableBashTool), since unlike the file tools above it isn't confined to reading/writing files — only enable it if you trust the connected model/server. Always asks for confirmation showing the exact command before running it; output is truncated if very long, and the command is killed after 60s.
• update_plan — create or update the visible task-list panel described in Chat above, for tracking multi-step or multi-file work. Gated behind llamanexus.enablePlanningTool (default on) — never touches disk, so it's allowed even in Plan Mode.
• run_agent — delegate a self-contained sub-task to this session's separately-configured agent server (see Server discovery above), for when this session's own server has no spare capacity for side-work. Off by default (llamanexus.enableAgentTool), since it dispatches your task/workspace context to a different machine and the agent can itself write files/run bash there. The agent runs its own bounded tool-calling loop (capped by llamanexus.maxAgentSteps, default 8) restricted to the same read/write/patch/list/bash tools — never update_plan or run_agent itself, so it can't recursively delegate — going through the exact same confirmations as the main model's own tool calls. Blocked entirely in Plan Mode. Shows as its own collapsible Agent note: the task first, then its nested Read/Write/Bash actions as they happen, then its final summary.

All file paths are resolved relative to (and confined to) your first workspace folder — the model can't read or write outside it. If the model emits a tool call as literal text (e.g. some models write <function=...> tags instead of a proper structured call), the extension detects and parses that as a fallback so it still works — checked in the reasoning/thinking text too if it's not found in the real reply, since some models/templates route it there instead — and write_file/patch_file/run_bash/run_agent are rejected the same way a real structured call would be while a session is in Plan Mode (see Chat above), so the fallback path can't be used to bypass it either.

Confirmations (writing/patching a file, running a bash command, ejecting a model, removing an RPC worker, proposing a plan, dispatching to an agent) show as an in-chat dialog — popping up above the composer in the same spot/style as the model list — rather than a native VS Code popup. Escape counts as denying the action, but clicking elsewhere (e.g. into the composer) does not — only an explicit button click or Escape answers the dialog. write_file/patch_file/run_bash/run_agent prompts also offer a "Confirm for this session" option: picking it approves the current call and skips the dialog for any further calls to that same tool for the rest of the session (tracked per tool, not persisted across reopening). Plan-approval prompts instead offer "Request changes...": picking it tells the model to hold off and wait for feedback, and focuses the composer so you can describe what you'd like different as your next message.

To guard against a model getting stuck in a loop, a tool-calling chain stops early if the model issues the exact same tool call (name + arguments) llamanexus.maxToolCallRepeats times in a row (default 3), or after llamanexus.maxToolChainDepth chained round-trips as a backstop (default 20) — both configurable in settings.

Hugging Face model search

LlamaNexus: Search Hugging Face for Models (also reachable via a "Search Hugging Face for models..." entry always shown at the bottom of the /model picker, seeded with whatever you'd typed) opens a panel that searches Hugging Face directly:

• Matching repos are listed collapsed by default — click one to load its quant files (cached after the first load).
• Quant files are grouped so a quant split across multiple shards (a common Hugging Face convention for large models) shows as a single entry with a combined size, rather than one row per shard file.
• A "✓ Downloaded" badge marks quants you already have (best-effort match against the currently loaded model list).
• The icon-only download button doubles as a cancel button once a download starts (it turns into a red square-with-X) — click it to abort. Downloading a multi-shard quant asks for confirmation first (states the total size and that it restarts the server once when done), then fetches every shard as one batch with a single combined progress bar and one restart at the end.
• Search/listing requests go straight to huggingface.co (not through the LlamaNexus proxy — downloading itself still does), so a gated or private repo needs its own auth: run LlamaNexus: Set Hugging Face Token and paste an access token from huggingface.co/settings/tokens. It's stored in VS Code's SecretStorage (OS keychain-backed), never written to settings.json or synced. LlamaNexus: Clear Hugging Face Token removes it. Public repos work fine with no token set.

Slash commands

Type / in the chat box for a command menu:

• /model — opens a picker of every model LlamaNexus reports. Each entry shows:
  • A dot that's green when the model is currently loaded on the server, gray otherwise.
  • An Eject button (only on loaded models) to unload it and free its memory — asks for confirmation first.
  • A current tag on whichever model is selected in settings.
  • Typing a name that isn't in the list but looks like repo:quant (e.g. Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M) or repo:filename (the exact .gguf file, e.g. Qwen/Qwen2.5-0.5B-Instruct-GGUF:qwen2.5-0.5b-instruct-q4_k_m.gguf) offers a Download "..." entry — picking it downloads the model via LlamaNexus, shows progress in a bar under the server list, and switches to it once ready.
  • A Search Hugging Face for models... entry is always shown at the bottom, opening the proper search panel (see Hugging Face model search above) instead of guessing a repo/quant string.
• /remove-model <model name> — opens the same live-filtered picker as /model, so you can search and pick a model instead of typing an exact name, but without the download/eject/Hugging-Face-search entries (only downloaded models can be removed). Picking one asks for confirmation - worded as permanent, since it deletes the model's file(s) from disk and its router.preset.ini entry, not just unloads it from memory like Eject. Requires LlamaNexus 0.1.21+.
• /ctx-size <size> — e.g. /ctx-size 8192. Changes the currently selected model's ctx-size via router.preset.ini and restarts the server to apply it, then refreshes the context bar's total.
• /worker <ip:port> (or /worker add <ip:port>) — e.g. /worker 192.168.0.120:50052. Adds an RPC worker to the proxy's manual roster and restarts the server to pick it up (see Server discovery above).
• /worker rm <ip:port> — e.g. /worker rm 192.168.0.120:50052. Removes a worker from the manual roster and restarts the server. Both forms report an error if issued while the server's in auto-discovery mode (roster's auto-managed) or, for rm, while no manual roster is active.
• /compact — manually summarizes the older part of the conversation right away, regardless of llamanexus.autoCompactThreshold (see Chat above).

Editor commands

• Explain Selection — right-click selected code → sends it to chat with an explain prompt.
• Fix / Improve Selection — right-click selected code → asks for a reviewed, corrected version.
• Ask About Selection — right-click, type a free-form question about the selected code.
• Generate Code from Comment/Selection — write a comment describing what you want, run the command, and it replaces/inserts the generated code inline (no chat panel needed).
• Insert Last Chat Response at Cursor — grabs the last assistant reply's first code block and drops it at your cursor.
• Select Model — same picker as /model, available from the command palette too.
• Search Hugging Face for Models — opens the search panel described above, available from the command palette.
• Set Hugging Face Token / Clear Hugging Face Token — see Hugging Face model search above.
• Check Connection Status — status bar indicator shows whether LlamaNexus is reachable; click to re-check.

Account

The Sessions sidebar shows real sign-in state - a Sign in button when logged out, or your username/email when logged in (click to log out) - backed by the same LlamaNexus: Log In / LlamaNexus: Log Out commands, also available from the command palette. Both authenticate with a LlamaNexus-Docs account - the same one used on the docs site and in WebUI. Requests go straight to LlamaNexus-Docs's public API (set llamanexus.accountApiBaseUrl first), not through the LlamaNexus proxy - the proxy has no involvement in accounts. The token is stored in VS Code's SecretStorage, the same as the Hugging Face token above, never a plain configuration setting; display info (email/username) is cached separately in globalState and re-validated on startup so an expired session doesn't linger in the sidebar.

Licensing

LlamaNexus: Set License Key / LlamaNexus: Clear License Key register this install against your account's plan - the key is stored in SecretStorage, same rationale as the account token above, never a plain llamanexus.* setting. A background LicenseHeartbeat (mirrors the server-discovery timer's shape) registers this install's globalState-persisted UUID once on activation and every 6h after, reusing llamanexus.accountApiBaseUrl for the call.

Opt-in only - no key set means no license API calls at all. A confirmed payment problem (an expired Trial, a past-due payment, or an invalid key) adds a note to the status-bar tooltip, shows exactly one warning popup per new occurrence, and blocks sending new chat messages until resolved - everything else (settings, past sessions, status bar) stays usable. Being over the plan's VS Code install-count limit (counted independently from WebUI installs on the same account) is a capacity issue, not a payment one, and stays a soft warning only. A plain connectivity failure stays silent for up to 7 days before showing anything, so a temporary outage doesn't produce a false warning.

Requirements

Run your LlamaNexus proxy first, e.g.:

./llamanexus serve -m /path/to/model.gguf --port 11434 --llama-port 8080 --jinja

--jinja is recommended if you want tool calling to work reliably — without it, llama-server falls back to minimal built-in template handling instead of the model's real chat template, which for some models (e.g. Qwen3-Coder) means tool calls can leak out as literal text instead of structured JSON on later turns in a conversation.

Settings

Grouped in the Settings UI under four categories (Extensions > LlamaNexus):

Connection

Model & Generation

Tool Calling

These four flags are the global defaults; the Controls sidebar's Tools section (see above) can override any of them per session.

Chat Appearance

Building / Installing locally

npm install
npm run build      # bundles to dist/extension.js
npm run package    # produces llamanexus-for-vscode-<version>.vsix

Then in VS Code: Extensions view → ... menu → Install from VSIX..., and pick the generated file.

Notes

• Streaming relies on LlamaNexus forwarding llama-server's SSE stream unmodified through /openai/v1/chat/completions, which is what the current proxy implementation does (including tools/tool_choice/stream_options fields — they pass through untouched).
• If your proxy's chat/completions endpoint expects a model field but you leave llamanexus.model empty, the request omits model entirely; make sure LlamaNexus/llama-server has a model already loaded in that case.
• Context size reporting is authoritative when your proxy's /openai/v1/models response includes context_length sourced from router.preset.ini (the real --ctx-size the router applied). It falls back to llamanexus.contextSize for models with no preset entry, or if the request fails.
• Token-usage numbers in the context bar come from the server's own reported usage when available; if the server doesn't report it for a given response (some servers omit it during tool calls, or don't support it at all), the extension estimates it client-side from character counts instead, and the bar shows a ~ prefix to indicate that.