BudgetAware

A VS Code coding assistant that prepares reviewable source changes under a visible output-token allocation. Work on small fixes in a separate snapshot, inspect the result and diff, then apply or discard the proposal. The same agent supports controlled research experiments. No model training and no hidden chain-of-thought access.

Quick start

Requires Node.js 20+ and VS Code 1.95+. In this directory:

npm.cmd ci --ignore-scripts
npm.cmd test
npm.cmd run benchmark:validate

Install budgetaware-0.4.2.vsix with Extensions > Install from VSIX, then reload VS Code. BudgetAware activates automatically after startup. Click BudgetAware: Ready in the status bar to check the environment, open the pilot guide, create a multi-file practice project, run a task, review changes, inspect results, or export metadata. It makes model requests only during a task you start.

0.4.2 makes workspace planning compact: attach stage shares to a first inspection action, keep the full action/result history, and show only the latest budget/plan feedback per request. All planning output is charged. Codex model selection reads the account's current visible catalog; the default is gpt-5.6-luna. See release and limitations. This is a combined product intervention, not a pure budget-awareness condition.

Version 0.4.0 offers strict output limits with compatible providers, or explicitly selected soft allocation with Codex. Workspace tasks request a revisable spending plan and receive feedback comparing actual output with planned stage allocations. The result screen shows both; stage shares are intentions, not enforced caps. An absent plan is not invented. This encourages economical work but does not guarantee savings or unchanged quality. See minimum product criteria and new pilot protocol.

0.4.1 requires a valid spending plan before a product task can edit/create files or finish. Rejected responses still consume the budget. Codex's native task-list updates are recorded as bookkeeping and do not stop a repair; commands, external tools and unknown activity still invalidate the response. Native lists never substitute for the spending plan. See product changes.

For development, press F5 using the BudgetAware Extension launch configuration. Choose BudgetAware: Run Benchmark Demo, then a task, condition, allocation, and provider. Codex has completed real coding demonstrations; the scripted mock needs no credentials and validates the pipeline only.

Use BudgetAware: Set API Key to store a Gemini/Anthropic key in VS Code SecretStorage. Environment variables GEMINI_API_KEY and ANTHROPIC_API_KEY are also supported. Never commit keys. This session's Anthropic account could list models but generation failed for insufficient credit.

The status bar shows measured remaining allocation and the most recent action category. After each run, a result screen separates the stop reason, independent evaluation, public checks, budget use, and changed files. It also records structured pilot feedback locally; metadata-only export excludes the optional private note. History reopens the same screen. Cancel from the progress notification.

Ollama and Codex are now supported in both Run Benchmark Demo and Run Coding Task. Ollama is listed first and is the CLI default: the installed llama3.1:8b runs locally without cloud API payment. Codex uses its saved login (ChatGPT on this machine), with gpt-5.6-luna by default and a visible-model selector. Older CLI discovery failures fall back to an editable model field. Defaults are stored in budgetaware.ollamaModel, codexModel, geminiModel, and anthropicModel. See provider setup and budget semantics.

Run Coding Task prepares changes in a separate snapshot of the first opened, trusted workspace. Save dirty files first. Your project stays unchanged until you apply the proposal. Select files to open read-only diffs. Applying uses normal VS Code edits: save the files or use Undo. A conflicting disk or unsaved editor change blocks application. Pending proposals survive reloads. Snapshots include at most 200 supported source files, 64 KB each, with hidden/build/dependency files excluded. New files must use supported types and existing directories. There is no shell tool. Optional .budgetaware-tests.json registers a pure-JavaScript suite, including bounded relative CommonJS/JSON modules:

{"path":"src/available.js","modules":true,"cases":[{"name":"empty","args":[[],[]],"expected":[]}]}

The entry must export a synchronous CommonJS function. Relative local JavaScript/CJS/JSON imports are evaluated in memory; the interpreter exposes no host filesystem, network, environment, or external packages. This prototype does not run arbitrary npm/Python/project test suites. Freeform completion has unknown independent success; benchmark mode uses separate held-out checks.

Reproducible runs

node dist/cli.js batch --provider mock --tasks clamp,slug --conditions CONTROL,INITIAL,RELATIVE,ADAPTIVE --budgets 200,2000 --repetitions 2 --max-runs 32 --out artifacts/new-mock-pilot
node dist/cli.js batch --provider gemini --tasks clamp,slug --conditions CONTROL,RELATIVE --budgets 2000,4000 --repetitions 2 --max-runs 16 --out artifacts/new-real-pilot
node dist/cli.js analyze --out artifacts/new-real-pilot
node dist/cli.js batch --provider ollama --tasks clamp --conditions CONTROL,RELATIVE --budgets 2000 --max-runs 2 --out artifacts/new-local-pilot
node dist/cli.js batch --provider codex --tasks clamp --conditions CONTROL,RELATIVE --budgets 2000 --max-runs 2 --out artifacts/new-codex-pilot
npm.cmd run test:extension
npm.cmd run package

Output directories must be new: the runner refuses reuse. Each run gets a clean, hashed fixture and fresh conversation. A batch manifest records the randomized schedule and configuration before execution. Provider errors stop the batch; existing data is preserved. Usage caps limit output, not input cost. Keep batches small.

Research conditions and measurement

CONTROL has no model-visible allocation; INITIAL receives the total once; RELATIVE receives remaining amount and percentage before each request; ADAPTIVE adds the previous observed action category without prescribing a workflow. All share enforcement and tool permissions.

Gemini budget = API-reported candidate tokens with thinking explicitly disabled. Anthropic budget = API-reported output tokens without extended thinking requested. Ollama budget = reported generated tokens with a requested generation limit. Codex budget = CLI-reported output tokens with a soft stop after each response; overshoot is possible and recorded. Scripted mock units are synthetic. Input usage is separate. Unknown usage stops generation and remains null. Forced stopping is distinct from natural finish and evaluator success.

There are 12 deterministic single-function tasks and three small multi-file practical tasks, each with failing starting code and passing reference checks. They validate the platform; they do not represent the full complexity of software engineering.

Evidence and documentation

See progress, architecture, research, literature, protocol, metrics, limitations, and adversarial review. Saved runs include config, exact model-visible messages, usage, ordered events, independent evaluation, and final source patch. Do not publish workspace transcripts without reviewing their contents.

Real and mock results are always kept separate. This is an exploratory research prototype; no hypothesis is confirmed. The completed 96-run Codex comparison and the separately audited 24-run local practical comparison are summarized in current findings. Their providers, tasks and budget enforcement differ; do not pool them.

School presentation and shortened pilot

The existing presentation is historical: it uses a balanced 8-run demonstration on one task, with four conditions and two soft allocations. The broader series has since been completed in artifacts/awareness-study; the original 34 completed runs and interrupted attempt remain unchanged in artifacts/school-study. The slides have not been updated to the 96-run results. See scope amendment, verified short-pilot results, presentation package, live demo, and practical application. The small demonstration does not establish an advantage from budget awareness.