AGENTEVAL Extension

Evaluate AI agents while you build them.

The extension is a thin client over the Python evaluation engine. It does not reimplement evaluators — it discovers agents, starts runs, and shows results.

Install (development)

1. Install the Python engine from the repo root:

pip install -e ".[dev]"

2. Open extension/ in VS Code / Cursor / Antigravity and press F5 (Extension Development Host), or:

cd extension
npm install
npm run compile

3. Commands:

• AGENTEVAL: Discover Agents
• AGENTEVAL: Run Evaluation
• AGENTEVAL: Compare Runs
• AGENTEVAL: Failure Autopsy
• AGENTEVAL: Evaluate Current Agent
• AGENTEVAL: Init Config

The sidebar talks to python -m agenteval serve on http://127.0.0.1:8765.

Icon

Use ../assets/logo.jpg as the marketplace icon (export a square PNG to media/icon.png before publishing).