Prompt Gym
A dedicated scratchpad for drafting and optimizing prompts before sending them to Copilot, ChatGPT, Claude, or any other LLM — with live token counting, waste detection, and one-click cleanup. Everything runs locally: no API key, no network requests, no model inference.
Screenshots
Issues tab — filler phrases flagged with one-click Remove buttons
Optimized tab — diff view showing exactly what was removed
Why High? tab — token composition breakdown
History tab — past prompts with Load into editor
Similarity warning — catches re-sent prompts before you waste tokens
What it does
| Signal |
How it works |
| Live token count |
gpt-tokenizer (o200k_base or cl100k_base) — offline, updates as you type |
| Filler phrase detection |
Regex against 30+ patterns ("basically", "I was wondering if you could", "thanks in advance") |
| Exact redundancy |
Sliding 6-word window detects verbatim repeated context |
| Semantic redundancy |
Jaccard similarity on stop-word-filtered sentence pairs catches paraphrased repetition |
| Structural issues |
Closing paragraph that repeats the opening; bullet preamble that echoes the list content |
| Oversized code blocks |
Fenced code blocks above a token threshold are flagged for trimming |
| One-click Remove |
Every removable flag has an inline Remove button — no need to find and delete manually |
| Optimized diff view |
Side-by-side Diff / Clean view shows exactly what was removed |
| Prompt history |
30-day history with similarity matching warns when you re-send near-duplicate prompts |
| Status bar counter |
Token count for the active file or selection, always visible at the bottom of VS Code |
| Right-click optimize |
Select text in any file → right-click → Prompt Gym: Optimize Selection |
Requirements
- VS Code 1.90 or later
- Node.js 18 or later
- npm 9 or later
Running locally
1. Clone and install
git clone <repository-url>
cd prompt-gym
npm install
2. Build
npm run build
This bundles src/extension.ts (and all imports) into dist/extension.js via esbuild. You should see:
dist/extension.js ~4.0mb
dist/extension.js.map
⚡ Done in ~200ms
3. Launch the Extension Development Host
- Open the
prompt-gym folder in VS Code:
code .
- Press F5 (or go to Run → Start Debugging).
- A new VS Code window opens — this is the Extension Development Host. All extension code runs here.
4. Open Prompt Gym
In the Extension Development Host window:
- Press
Cmd+Shift+P (Mac) or Ctrl+Shift+P (Windows/Linux)
- Type Prompt Gym: Open Rehearsal Space and press Enter
The panel opens as a full editor tab with a textarea on the left and a live analysis sidebar on the right.
Using the extension
Core workflow
- Type or paste your prompt into the textarea.
- The sidebar updates live (350 ms debounce) showing:
- Token count gauge (green → yellow → red)
- Number of flags found
- Estimated savable tokens
- Token count after cleanup
- Issues tab — each flag has a type, message, suggestion, and a Remove button for one-click fixes.
- Why High? tab — stacked bar chart breaking down tokens by narrative / code / filler / repeated content, with plain-English explanations.
- Optimized tab — diff view (strikethrough removals) and clean view of the prompt after all removable flags are applied. Copy with the Copy optimized button.
- History tab — past sent prompts with timestamps and token counts. Click Load into editor to bring one back.
Copying a prompt
- Copy prompt — copies the current draft as-is and saves it to history.
- Copy optimized — copies the cleaned version (filler and redundancy removed) and saves it to history.
Either button is the signal that you're sending this prompt to an LLM. The history store uses this to power the similarity warning on future prompts.
Optimize selection (right-click)
- Open any file in VS Code.
- Select the text you want to optimize.
- Right-click → Prompt Gym: Optimize Selection.
- Choose: Replace in editor, Copy to clipboard, or Open in Prompt Gym panel.
Status bar token counter
The status bar (bottom-right) shows:
- Token count for the active document while no panel is open.
(sel) suffix when text is selected — shows selection-only count.(gym) suffix while the Prompt Gym panel is active — shows the panel's count.
Click the status bar item to open the Prompt Gym panel.
Model selector
The dropdown in the toolbar switches the tokenizer and context window reference:
| Model |
Tokenizer |
Notes |
| GPT-4o, GPT-4.1, o3, Copilot |
o200k_base |
Exact |
| GPT-4, GPT-3.5 |
cl100k_base |
Exact |
| Claude Sonnet/Haiku/Opus 4 |
cl100k_base |
~95% accurate |
| Gemini 2.5 Flash/Pro |
cl100k_base |
~95% accurate |
Settings
Open Settings (Cmd+,) and search for Prompt Gym:
| Setting |
Default |
Description |
promptGym.tokenizerModel |
o200k_base |
Tokenizer encoding. o200k_base ≈ GPT-4o family. cl100k_base ≈ GPT-4/3.5 family. Overridden by the model selector in the panel. |
promptGym.tokenWarningThreshold |
2000 |
Token count above which a threshold warning flag appears. |
promptGym.pastedCodeTokenThreshold |
300 |
Fenced code blocks above this many tokens are flagged as oversized. |
Watch mode (auto-rebuild on save)
Instead of re-running npm run build after every change, use watch mode:
npm run watch
esbuild rebuilds in ~200 ms on each save. After a rebuild, reload the Extension Development Host window with Cmd+Shift+P → Developer: Reload Window.
Running the test suite
The tests run directly on the compiled analyzer — no VS Code instance required.
Run all tests
node test-runner.mjs
The runner builds a test-only bundle (dist/analyzer-test.cjs) and executes 78 tests across 9 suites. Output goes to stdout and results are written to eval.md.
══ countTokens ══════════════════════════════════════════════
✓ empty string → 0 tokens
✓ single word has at least 1 token
...
══ Results: 78/78 passed, 0 failed ══
Eval written to /path/to/prompt-gym/eval.md
Test suites
| Suite |
Tests |
What's covered |
countTokens |
9 |
Empty input, unicode, 10 k-word corpus, both tokenizers agree within 20% |
findFillerPhrases |
10 |
Single-word and multi-word patterns, case-insensitive, offset validity, false-positive check |
findRedundantContext |
4 |
Exact repetition, no false positives on unique sentences, short-text guard |
findSemanticRedundancy |
7 |
High word-overlap sentences, adjacent-sentence skip, very short sentence guard |
findStructuralIssues |
5 |
Closing paragraph repeats opening, bullet preamble echo, paragraph count guard |
findOversizedCodeBlocks |
5 |
Threshold enforcement, multi-block selectivity, tokenImpact > 0 |
generateOptimizedPrompt |
9 |
All removable flag types, no double-spaces, capitalisation, idempotent on clean input |
analyzePrompt (integration) |
13 |
Output shape, breakdown arithmetic, flag sort order, model fallback |
| Edge cases |
16 |
Null bytes, emoji, whitespace-only, 10 k-word perf (< 2 s), overlapping flags, unknown modelId |
Reading eval.md
After each run, eval.md is overwritten with a full report — pass/fail per test, failure message if any, and a summary table. Check it in after a feature change to capture the baseline.
Type checking
npx tsc --noEmit
No output means clean. The project uses strict TypeScript — all analyzer exports are fully typed.
Project structure
prompt-gym/
├── src/
│ ├── extension.ts # VS Code activation, commands, status bar, webview lifecycle
│ ├── analyzer.ts # All token-counting and flag-detection logic (no VS Code deps)
│ ├── history.ts # 30-day prompt history store (VS Code globalState)
│ └── webviewContent.ts # Full webview HTML/CSS/JS as a template string
├── dist/
│ └── extension.js # esbuild output — what VS Code actually loads
├── test-runner.mjs # Self-contained test runner (78 tests, no test framework)
├── eval.md # Auto-generated test results (overwritten on each run)
├── esbuild.config.js # Build config
├── tsconfig.json
└── package.json
analyzer.ts has zero VS Code dependencies — it's plain TypeScript that can be tested with bare Node. This is intentional: all detection logic stays independently testable without spinning up an Extension Development Host.
Packaging a VSIX (local install)
To install without publishing to the marketplace:
npm install -g @vscode/vsce
vsce package
This produces prompt-gym-0.0.1.vsix. Install it via:
- VS Code:
Cmd+Shift+P → Extensions: Install from VSIX → select the file
- CLI:
code --install-extension prompt-gym-0.0.1.vsix
What's intentionally not in this version
- No AI-based rewriting — keeps it free, fast, and offline. Could be added as an opt-in command using
vscode.lm (VS Code's built-in LLM API) without requiring a separate API key.
- No dollar cost estimates — pricing tables go stale fast; Tokenometer and Tokenlint already cover this well.
- No auto-injection into Copilot Chat — Copilot's input box is not extensible by third-party extensions. Copy/paste is the intentional handoff.
- No Claude API tokenizer — Claude uses a proprietary tokenizer not available as an npm package. The cl100k_base approximation is within ~5% for typical prompts.
