LlamaNexus for VS Code

Chat with, explain, and edit code using your local LlamaNexus proxy (your Go proxy in front of llama.cpp), directly inside VS Code — no Open WebUI needed.

It talks to LlamaNexus's OpenAI-compatible endpoints:

• POST {baseUrl}/openai/v1/chat/completions (streaming SSE, with tool-calling support)
• GET {baseUrl}/openai/v1/models (includes each model's real context size and whether it's currently loaded)
• GET {baseUrl}/status (connection check, plus current RPC worker mode/roster)
• POST {baseUrl}/api/set-ctx-size (change a model's ctx-size and restart to apply it)
• POST {baseUrl}/api/eject-model (unload a model from the server)
• POST {baseUrl}/api/rpc/mode, /api/rpc/workers, /api/rpc/workers/remove (toggle single-node/RPC mode and add/remove RPC workers)
• POST {baseUrl}/ollama/api/pull (download a not-yet-present model, streaming progress)

Features

Sessions

Click the LlamaNexus icon in the Activity Bar for a Sessions sidebar: "+ New session", a live-filtering "Search sessions..." box, and a list of every session with rename (pencil) and delete (trash bin) controls. Sessions are per-workspace and persist (including full history, response stats, and any reasoning/thinking text) across window reloads — reopening one replays its prior conversation exactly as it looked before. Selecting or creating a session opens its chat in its own editor tab.

Chat

LlamaNexus: Open Chat (Ctrl+Alt+L / Cmd+Alt+L), or click a session in the Sessions sidebar — each session is its own editor-tab chat with:

• Streaming responses, labeled with whichever model actually answered.
• Thinking/reasoning models (anything llama-server streams reasoning_content for) get their reasoning shown in a collapsible "Thinking..." section, separate from the final answer and not resent to the model on later turns.
• Markdown-ish rendering: #/## headers, **bold**, *italic*/_italic_, `inline code`, and -/* lists, plus full fenced code blocks.
• Copy and Insert buttons on every code block. Insert drops the code at your cursor in the last editor you had focus in (works even though focus is on the chat tab when you click it).
• User messages are shown in their own card; tool-call/tool-result activity shows as collapsible system notes (collapsed by default — click to expand).
• Attach one or more files to a message (the + button) — each shows as a removable chip while composing and as a read-only chip on the sent message, both live and after reopening the session. Attached content is folded into the message sent to the model (200 KB limit per file; binary files are rejected).
• A context-usage bar (used vs. total tokens) that fills as the conversation grows, sized from the model's real ctx-size (see Model tools below) or your manual llamanexus.contextSize setting as a fallback.
• Response stats (prompt/response/total tokens, elapsed time, tokens/sec) under each assistant reply.

Model tuning sidebar

A collapsible sidebar on the right edge of the chat (click the ⚙ strip to expand) mirrors Open WebUI's per-chat "Advanced Params": Temperature, Top K, Top P, Min P, Frequency Penalty, Presence Penalty, Repeat Penalty, Repeat Last N, Mirostat (+ Tau/Eta), Seed, Max Tokens, and Stop Sequences. Each field has its own enable checkbox — unchecked fields are left out of the request entirely (the server/llamanexus.temperature/llamanexus.maxTokens defaults apply); checking one and setting a value makes it stick for every message from then on. Enabled values persist with the session, independent of the global settings, and are restored the next time that session is reopened.

Server discovery

A row under the context bar lists every LlamaNexus serve instance found on your LAN (requires LlamaNexus 0.1.5+, which broadcasts a UDP heartbeat on llamanexus.discoveryPort, default 50050). Each session picks its own server independently — click a row to select it for the current session; the choice persists with the session and is restored (if that server's still online) the next time it's reopened. Until a server is selected, the composer is disabled with a prompt to choose one.

The selected server's row expands (click it) to show its RPC worker mode/roster, when running in --rpc/--discovery mode:

• A button to toggle between single-node mode and RPC mode (remembers whichever RPC source — discovery or a manual roster — was last active).
• Per-worker remove (manual mode only; the roster is read-only while auto-discovery is active).
• The /worker <ip:port> slash command to add a worker.

Model tools (function calling)

When the connected model/server supports tool calling, LlamaNexus can:

• read_file — read a file in your open workspace.
• patch_file — make a targeted edit by replacing one exact snippet of a file's current text, without rewriting the whole file. Preferred over write_file for existing files, since it avoids re-emitting entire files as one giant argument (which risks hitting the token cap on large files).
• write_file — create a new file or fully rewrite one. Always asks for a modal confirmation before touching disk, showing what would change.
• list_files — list a directory's contents.

All file paths are resolved relative to (and confined to) your first workspace folder — the model can't read or write outside it. If the model emits a tool call as literal text (e.g. some models write <function=...> tags instead of a proper structured call), the extension detects and parses that as a fallback so it still works.

To guard against a model getting stuck in a loop, a tool-calling chain stops early if the model issues the exact same tool call (name + arguments) llamanexus.maxToolCallRepeats times in a row (default 3), or after llamanexus.maxToolChainDepth chained round-trips as a backstop (default 20) — both configurable in settings.

Slash commands

Type / in the chat box for a command menu:

• /model — opens a picker of every model LlamaNexus reports. Each entry shows:
  • A dot that's green when the model is currently loaded on the server, gray otherwise.
  • An Eject button (only on loaded models) to unload it and free its memory — asks for confirmation first.
  • A current tag on whichever model is selected in settings.
  • Typing a name that isn't in the list but looks like repo:quant (e.g. Qwen/Qwen2.5-0.5B-Instruct-GGUF:Q4_K_M) or repo:filename (the exact .gguf file, e.g. Qwen/Qwen2.5-0.5B-Instruct-GGUF:qwen2.5-0.5b-instruct-q4_k_m.gguf) offers a Download "..." entry — picking it downloads the model via LlamaNexus, shows progress in a bar under the server list, and switches to it once ready.
• /ctx-size <size> — e.g. /ctx-size 8192. Changes the currently selected model's ctx-size via router.preset.ini and restarts the server to apply it, then refreshes the context bar's total.
• /worker <ip:port> — e.g. /worker 192.168.0.120:50052. Adds an RPC worker to the proxy's manual roster and restarts the server to pick it up (see Server discovery above).

Editor commands

• Explain Selection — right-click selected code → sends it to chat with an explain prompt.
• Fix / Improve Selection — right-click selected code → asks for a reviewed, corrected version.
• Ask About Selection — right-click, type a free-form question about the selected code.
• Generate Code from Comment/Selection — write a comment describing what you want, run the command, and it replaces/inserts the generated code inline (no chat panel needed).
• Insert Last Chat Response at Cursor — grabs the last assistant reply's first code block and drops it at your cursor.
• Select Model — same picker as /model, available from the command palette too.
• Check Connection Status — status bar indicator shows whether LlamaNexus is reachable; click to re-check.

Requirements

Run your LlamaNexus proxy first, e.g.:

./llamanexus serve -m /path/to/model.gguf --port 11434 --llama-port 8080 --jinja

--jinja is recommended if you want tool calling to work reliably — without it, llama-server falls back to minimal built-in template handling instead of the model's real chat template, which for some models (e.g. Qwen3-Coder) means tool calls can leak out as literal text instead of structured JSON on later turns in a conversation.

Settings

Building / Installing locally

npm install
npm run build      # bundles to dist/extension.js
npm run package    # produces llamanexus-for-vscode-<version>.vsix

Then in VS Code: Extensions view → ... menu → Install from VSIX..., and pick the generated file.

Notes

• Streaming relies on LlamaNexus forwarding llama-server's SSE stream unmodified through /openai/v1/chat/completions, which is what the current proxy implementation does (including tools/tool_choice/stream_options fields — they pass through untouched).
• If your proxy's chat/completions endpoint expects a model field but you leave llamanexus.model empty, the request omits model entirely; make sure LlamaNexus/llama-server has a model already loaded in that case.
• Context size reporting is authoritative when your proxy's /openai/v1/models response includes context_length sourced from router.preset.ini (the real --ctx-size the router applied). It falls back to llamanexus.contextSize for models with no preset entry, or if the request fails.
• Token-usage numbers in the context bar come from the server's own reported usage when available; if the server doesn't report it for a given response (some servers omit it during tool calls, or don't support it at all), the extension estimates it client-side from character counts instead, and the bar shows a ~ prefix to indicate that.