SimTooReal for VS Code
Author, train, and monitor robotics reinforcement-learning projects without leaving VS Code —
the describe → generate → train → monitor loop, powered by the simtooreal CLI and your
in-IDE Claude Code model.
Features
- SimTooReal sidebar — three live tree views:
- Projects — your platform projects, with inline Push / Train actions.
- Runs — training run history with live status icons; click to open the dashboard.
- Templates — starter robot/scene/RL templates; create a project from one in a click.
- Run dashboard (webview) — live status, best-reward sparkline, iteration progress; polls
while a run is active.
- Status bar — current active run (iteration + reward) at a glance.
- Streaming logs —
Stream Logs opens a terminal following the run output.
@simtooreal chat participant — describe a robot + task in the chat panel; the in-IDE
model recommends a template and generates the Isaac Lab env config / reward / train.py,
then surfaces buttons to push and train.
How it works (security)
The extension carries no business logic. Every platform action runs the simtooreal CLI
with --json; authentication and all proprietary logic live behind the CLI and the server it
talks to. The extension is
presentation only.
Requirements
- The
simtooreal CLI on PATH (or set simtooreal.cliPath):
npm i -g simtooreal-cli
- Sign in: Command Palette → SimTooReal: Sign In (or
simtooreal-cli login).
AI agent (MCP-first, Claude Code by default)
Code generation is driven by an AI agent through the SimTooReal MCP server,
not by VS Code's chat model. Install the MCP server once so agents can launch it:
npm i -g @simtooreal/mcp-server # provides the `simtooreal-mcp` binary
Then Command Palette → SimTooReal: Set Up AI Agent (MCP) (or the sidebar plug icon) writes
the right config for your agent:
Agent (simtooreal.defaultAgent) |
Config written |
claude-code (default) |
.mcp.json in the workspace root |
cursor |
.cursor/mcp.json |
codex |
~/.codex/config.toml ([mcp_servers.simtooreal]) |
vscode-chat |
uses the built-in @simtooreal participant (Copilot models) instead |
Create with AI registers the MCP server for your default agent, copies a starter prompt to
the clipboard, and opens the agent — paste the prompt and it scaffolds → generates → pushes →
trains via the MCP tools. Reopen/reload the agent after first registration so it picks up the tools.
Settings
| Setting |
Default |
Description |
simtooreal.cliPath |
simtooreal-cli |
Path to the CLI binary. |
simtooreal.defaultAgent |
claude-code |
Which agent "Create with AI" uses (MCP). |
simtooreal.pollIntervalSeconds |
5 |
Dashboard / status-bar refresh interval. |
Agent-native alternative
For agents that prefer native tool-calls (Claude Code, Codex, Cursor), register the
SimTooReal MCP server — same lifecycle, exposed as MCP tools.
Develop
cd extension && npm install && npm run build
# Press F5 in VS Code to launch an Extension Development Host.
