Project Voice

Project Voice is a VS Code extension for holding a voice call with a codebase. It supports concise explanations of selected code and multi-file guided tours. Deepgram handles transcription and speech, while the user's installed Claude Code or Codex CLI explores the workspace in read-only mode.

Development

Prerequisites: Node.js 20+, npm, VS Code, and at least one supported CLI (claude or codex).

npm install
npm run verify

Open this folder in VS Code and press F5 with Run Extension selected. The Extension Development Host opens with whatever folder you have open — pick any project you'd like to try it against. Run Project Voice: Start Call from the Command Palette. The sidebar appears and a loopback-only audio companion opens in VS Code's integrated Browser.

For the microphone human check, start the call. Allow the localhost site's microphone request in the companion the first time, confirm the level meter moves, then use Stop listening in the sidebar and confirm the meter returns to zero. Click Play test tone in the companion for the playback check. If the site prompt was previously denied, use Browser: Manage Permissions in the Command Palette, allow Microphone for http://127.0.0.1:43117, and reload the companion tab. The macOS permission for Visual Studio Code must also remain enabled.

The companion exists because extension sidebar webviews do not participate in VS Code's microphone permission service. It binds only to 127.0.0.1:43117, uses a random session path plus same-origin WebSocket validation, and never exposes audio devices to the extension host. The stable port lets VS Code remember the localhost site permission across restarts.

Phase C: Deepgram transcription

Run Project Voice: Set Deepgram API Key, or use Set Deepgram API key in the sidebar, before speaking. The key is stored in VS Code SecretStorage; the ignored .env file is not read at runtime. The extension exchanges the key for a short-lived Deepgram access token on each listening turn, so the persistent key is never sent to the browser companion. The key must have enough permissions to use Deepgram's /v1/auth/grant endpoint.

Start a call and Project Voice begins listening automatically when Deepgram is configured. Once the microphone is live, focus returns to the code editor while the required companion remains open in the background. Deepgram endpointing finalizes each spoken utterance while capture remains active, so you can ask another question or use a tour command without restarting the microphone. Stop listening explicitly ends capture. Empty audio chunks are not transmitted, audio produced while the temporary token is being minted is buffered, and every recorder/socket/timer is closed on stop or call teardown.

Phase D: Claude Code and Codex answers

Project Voice detects the installed claude and codex executables when the extension activates. Auto prefers Claude Code and falls back to Codex. Run Project Voice: Select CLI Provider, or use Select CLI provider in the sidebar, to choose one explicitly. The choice persists in VS Code workspace-independent extension state.

Both CLIs use the user's existing sign-in; the extension never reads another application's credentials. Codex runs through codex exec in its read-only sandbox. Claude Code runs in non-interactive mode with only Read, Glob, and Grep, and permission escalation is disabled. The CLI process is cancelled when a new answer supersedes it or the call ends.

Use Project Voice: Select Response Speed, or Select response speed in the sidebar, to choose a per-user profile. The selection is stored in VS Code extension global state and applies to subsequent requests:

• Fast enables Codex Fast mode with low reasoning effort; Codex documents roughly 1.5× model speed with higher credit consumption. Claude Code uses low effort.
• Balanced keeps Codex on the standard service tier with low reasoning effort for bounded tours. Claude Code uses medium effort.
• Deep uses the standard Codex tier and high reasoning effort. Claude Code also uses high effort.

The speed preference does not edit the user's global Codex or Claude configuration. Fast mode depends on the selected Codex model, sign-in type, and account availability.

The extension remembers the latest text-editor file, cursor line, selected line range, and up to 16,000 characters of selected text before focus moves to the audio companion. When listening starts it snapshots that remembered context. Once Deepgram finalizes the transcript, Project Voice sends the snapshot and the Phase-2 narration prompt to the selected CLI. The spoken-style text answer appears in both the audio companion and the sidebar.

To test Phase D, open a file in the Extension Development Host, select a useful block, and start the call. After the sidebar says listening, ask “what does this do?” and pause briefly for endpointing. Wait for the answer card. Its CLI status should name Claude Code or Codex, briefly show Thinking…, then display a short prose answer grounded in the selection. Test the other adapter with Project Voice: Select CLI Provider if both CLIs are installed.

Phase E: spoken answer and audio-synchronized highlight

The completed CLI answer is divided at sentence boundaries and streamed to Deepgram Aura-2 TTS through a browser-owned WebSocket. The companion receives raw 24 kHz linear PCM and plays it through an AudioWorklet. The persistent Deepgram key remains in SecretStorage; the browser receives only a fresh short-lived token for each spoken answer.

The AudioWorklet is the master clock. It reports the first rendered audio frame to the extension, and only then does the extension open the captured file with preserveFocus: true, reuse a preview tab, and highlight the original selection. The highlight clears when playback ends or is interrupted. Starting another listening turn interrupts current speech and clears its highlight.

To test Phase E, select code, start the call, and ask a question. The sidebar should progress from Voice: synthesizing to Voice: playing. Confirm you hear the answer and that the selection highlight appears when speech begins, not while the CLI is still thinking. If Chromium blocks Web Audio, click Play test tone once in the companion and repeat the question.

Milestone 2: guided code tours

Ask a broad implementation question without selecting code, for example, “How does the new Analysis API work?” Project Voice asks the selected CLI for a schema-validated tour of up to six real workspace locations. Claude Code uses --json-schema; Codex uses --output-schema inside its read-only sandbox. Absolute paths, workspace traversal, malformed ranges, duplicate stops, and invalid plans are rejected before any file opens.

Each stop is sent to Deepgram separately. When the first PCM frame actually plays, Project Voice focuses the corresponding preview tab, relocates a stale range by symbol when possible, and highlights the code. Playback completion advances to the next stop. The sidebar shows progress, files, symbols, sample provenance, and Previous, Pause, Repeat, Next, Go deeper, and Stop controls.

While narration is playing, the continuous microphone recognizes only bounded navigation commands: “pause,” “resume,” “next,” “back,” “repeat,” “go deeper,” and “stop tour.” Other speech is ignored to prevent speaker echo from becoming a question. “Go deeper” creates a nested tour rooted at the current stop, then returns to the following stop in the parent tour.

To test the milestone, open the Extension Development Host against a project with several related files, start a call, and ask a broad implementation question about it. Confirm that the sidebar displays a guided tour, the first code tab receives focus only when speech begins, subsequent stops cross real files, and voice or sidebar navigation cancels the old audio before moving. Repeat once with Claude Code and once with Codex if both are installed.

Inline input/output previews

Select a line, place the cursor on a call or expression, and ask, “What can be the possible input values and output after running this line?” Project Voice asks the selected CLI for up to three schema-validated cases, then displays them as a temporary blue information panel directly after the target line. Hover over the panel for the complete cases, evidence source, and assumptions. The same explanation is shown in the Answer card and narrated aloud.

The preview is evidence-aware. Observed means a test, fixture, or call site directly demonstrates the case and the repository source is shown. Inferred means the result follows from inspected code. Synthesized means the example is illustrative. Project Voice never executes arbitrary workspace code for this preview, so it does not describe inferred or synthesized values as captured runtime output.

The decoration disappears after 30 seconds, when another question starts, when another preview replaces it, or when the call ends. Unsafe paths and stale line numbers are rejected before a decoration is displayed.

To test it, save a code file, select a function call or place the cursor on it, and ask the example question above. Confirm a differently colored inline panel appears beside the line and the editor hover identifies the evidence level. Then ask another question and confirm the old panel disappears.

Milestone 3: reviewable code edits

Start an edit request with an explicit action, for example, “Please add validation for an empty project name” or “Could you refactor this function to remove the duplicate lookup?” Project Voice asks the selected CLI for a schema-validated proposal while the CLI remains read-only. The proposal may modify up to six existing workspace files. New, renamed, moved, and deleted files are intentionally not supported in this milestone.

Before displaying anything, the extension verifies safe workspace-relative paths, resolves symlinks, rejects targets outside the workspace, requires exact complete-line source matches, and rejects overlapping or stale edits. Save files before requesting a proposal. Project Voice opens the first change with VS Code's native diff editor and narrates what changed and why.

The sidebar lists each proposed file. Review one file at a time, then use Apply file or Reject file. Applying changes only the current editor buffer and does not save it automatically. You can also say “apply this file,” “reject this file,” “show the next file,” “show the diff,” or “cancel proposal.” Project Voice deliberately does not accept “apply all”; every file requires an explicit decision.

To test the milestone, open and save a code file in the Extension Development Host, start the call, and make an explicit edit request. Confirm that a native diff opens before narration, the original file remains unchanged while the proposal is pending, Reject file leaves it unchanged, and Apply file updates only that file as an unsaved editor change. Change the source after a proposal opens and confirm that applying is rejected as stale. Repeat once with Claude Code and once with Codex if both are installed.

Milestone 4: agent-routed edit tasks

Edit requests now route through a dedicated task-agent session. Project Voice makes a bounded decision for every request:

• A first task, an unrelated task, or “spawn an agent to update this code” starts a fresh task session.
• An anaphoric follow-up such as “just spawn a different agent which can do this” starts a fresh task session and includes the previous voice request and answer as its handoff context.
• “Continue the existing agent and update this code,” a follow-up such as “also update the same loader,” or another request against the same file continues the compatible task session.
• Broad feature, architecture, workflow, pipeline, or multi-file requests start fresh and request one worker agent for repository exploration. With Auto, Project Voice selects Codex when it is available for this delegated path; an explicit CLI selection is always respected.

The sidebar shows Fresh task agent or Existing agent session, whether worker delegation was requested, and the reason for the routing decision. Main voice-answer context and edit-task context use separate CLI threads, so an implementation task does not pollute the conversational thread.

A spawned agent remains read-only. For Codex, Project Voice enables the CLI's agent capability for the fresh task and directly asks for one worker when delegation is justified. Claude Code receives a fresh isolated read-only task session; Project Voice does not assume an unsupported Claude subagent interface. In both cases, the result is still a schema-validated proposal. No source file changes until the user explicitly applies each file in VS Code.

To test routing, make an edit request and confirm the proposal card says Fresh task agent. Then say “also update the same code” and confirm it says Existing agent session. Finally say “spawn an agent to update this feature across the codebase.” With Auto and Codex installed, the card should show Fresh task agent · worker delegation requested and the CLI status should identify Codex. Review and apply or reject each proposed file as usual.