PromptForge
AI-powered prompt engineering copilot for VS Code
PromptForge is a VS Code extension that helps you write, test, and refine prompts for Large Language Models — with an AI co-writer that knows the best practices of each model.
Features
Co-Writer with Model-Specific Best Practices
Write prompts with an AI assistant that adapts its suggestions based on the target model:
- Claude (Anthropic): XML tag structure, role-first system prompts
- GPT-5 / o-series (OpenAI): Markdown headers, concise directives
- Gemini (Google): Declarative instructions, thinking level control
Dynamic Variable Templates
Use {{variable}} and {{variable:default_value}} syntax in your prompts. PromptForge auto-detects variables and generates input forms.
Test Runner with Semantic Scoring
Test your prompts against live models, compare outputs with desired results, and get a semantic similarity score. Refine prompts iteratively based on real outputs.
A/B Testing
Compare two prompt versions side by side with the same inputs. See cost, latency, and output differences at a glance. Up to 20 inputs per test.
Call Replay
Capture API calls and replay them later with different prompts. Compare original vs. new outputs with a visual diff.
Cost Tracking
Track token usage and costs per model with 4-decimal precision. Supports 10 models across 4 providers.
Code Snippet Export
Generate ready-to-use code in TypeScript, Python, or cURL for Anthropic, OpenAI, and Google APIs (9 combinations).
Prompt Library
Save, version, search, and export your prompts. Up to 15 versions per prompt with automatic archiving.
Supported Models
| Provider |
Models |
| Anthropic |
Claude Opus 4.6, Claude Sonnet 4.6, Claude Haiku 4.5 |
| OpenAI |
GPT-5.4, GPT-5.4 Mini, o3, o4-mini |
| Google |
Gemini 3.1 Pro, Gemini 2.5 Flash, Gemini 3.1 Flash Lite |
| OpenRouter |
Auto (fallback) |
Getting Started
- Install the extension from the VS Code Marketplace
- Open Command Palette (
Ctrl+Shift+P) and run PromptForge: Configure API Keys
- Enter your API key for at least one provider
- Press
Ctrl+Alt+N to create your first prompt
Keyboard Shortcuts
| Shortcut |
Command |
Ctrl+Alt+N |
New Prompt |
Ctrl+Alt+T |
Test Prompt |
Ctrl+Alt+L |
Open Library |
Requirements
- VS Code 1.100.0 or later
- At least one API key (Anthropic, OpenAI, Google, or OpenRouter)
Extension Settings
| Setting |
Default |
Description |
promptforge.defaultModel |
claude-opus-4-6 |
Default model for new prompts |
Data Storage
- API Keys: Stored securely in VS Code SecretStorage (never in files)
- Prompts: Saved as JSON in
.promptforge/prompts/ (add to .gitignore)
- Replays: Saved as JSON in
.promptforge/replays/
Tech Stack
- TypeScript (strict mode)
- React 18 + Tailwind CSS (Webview UI)
- Zustand (state management)
- esbuild (bundler)
- Vitest (testing)
License
MIT
