SOFEE VS Code Extension
Execute SOFEE benchmark eval runs from VS Code, Cursor, or IBM Bob (Remote-SSH on the eval host).
v0.2.19 — Agent pack status in dashboard
- Dashboard section 5 — shows whether Cline rules and skills are materialized in the project and deployed in the open workspace, with counts and Install / Refresh action
- Sofee-Eye tab — agent pack status table (rules, skills, project path, central pack reference)
v0.2.18 — Matrix-level agent pack opt-out
- Eval auto-deploy opt-out — per-matrix checkbox in Sofee-Eye (Eval Matrices → "Skip agent skills & rules deploy"); runs inherit
skip_agent_pack_deploy from the matrix via the eval run API
- Global opt-out unchanged —
sofeeEvals.skipAgentPackDeploy still skips deploy for all runs
v0.2.17 — Agent skills & rules deploy
- Install / Refresh Agent Skills & Rules — copies
projects/{uid}/.clinerules/ and .cline/skills/ into the open Cline workspace (Cline profile only)
- Eval auto-deploy — after worktree creation, deploys the project's agent pack before MCP apply (opt out: matrix setting or
sofeeEvals.skipAgentPackDeploy)
- Projects root — optional
sofeeEvals.projectsRoot (defaults from eval runtime repo_open_path/projects)
v0.2.16 — Reliable Cline MCP toggle
- MCP ON — backs up Cline
cline_mcp_settings.json, disables all MCP servers, enables only the eval server (from SOFEE config snapshot or extension settings)
- MCP OFF — disables all Cline MCP servers for a true no-MCP baseline
- Restore — original Cline MCP settings restored after each run
- Cline path detection — auto-detects installed Cline extension globalStorage folder
- MCP apply deferred until eval worktree is open (avoids Cline overwriting settings on folder switch)
v0.2.0 — Dashboard + multi-IDE
- SOFEE sidebar (activity bar) — connection status, settings, projects, pending runs
- Auto-detect IDE — Cursor, IBM Bob, Cline, or VS Code
- MCP toggle writes to
.vscode/mcp.json, .cursor/mcp.json, .bob/mcp.json, or Cline's global cline_mcp_settings.json as appropriate
- Agent adapters — Cline/Bob automated test server; Cursor/Bob/VS Code manual clipboard mode
Install
From Marketplace (recommended): Extensions → search SOFEE → Install.
From VSIX (eval host / offline):
code --install-extension sofee-evals-vscode-0.2.5.vsix
On a Sofee-Eye dev or deploy host, prefer the automated script (also run by hot-load-for-dev.sh and build-deploy-all.sh):
bash /home/aiadmin/sofeei/sofee-eye/scripts/build-install-sofee-evals-vscode.sh
Then Developer → Reload Window in VS Code/Cursor.
Quick setup
- Open SOFEE in the activity bar (left sidebar)
- Paste Auth Token from Sofee-Eye browser login → Save & connect
- Confirm green Connected status
- Click Run on a pending eval run (or enable auto-queue)
Settings
| Setting |
Description |
sofeeEvals.apiBaseUrl |
Sofee-Eye backend (e.g. http://10.127.12.108:8000) |
sofeeEvals.authToken |
JWT from Sofee-Eye login |
sofeeEvals.ideProfile |
auto, vscode, cursor, bob, or cline |
sofeeEvals.sofeeMcpUrl |
MCP URL for MCP ON runs without a snapshot (public Traefik /mcp URL on eval hosts) |
sofeeEvals.sofeeMcpServerName |
Cline MCP server key when no snapshot (e.g. Sofee 10xOne — must match Cline UI name) |
sofeeEvals.autoQueue |
Poll and execute pending runs automatically |
Build
cd extensions/sofee-evals-vscode
npm install
npm run compile
npm run package
From the Sofee-Eye repo, build + install in one step:
bash scripts/build-install-sofee-evals-vscode.sh
This is also invoked automatically by scripts/hot-load-for-dev.sh and scripts/build-deploy-all.sh.
See MARKETPLACE_PUBLISH.md for Marketplace release tagging and CI.
