PromptyQ

Get better at building software with AI agents, not just faster at generating it.

PromptyQ puts Codex, Claude Code and Gemini in a chat inside VS Code and quietly measures how you work with them: how clear your prompts are, whether you actually review what the agent wrote, whether you test it, whether the change is secure and well designed. Every day you get a report per project with scores, what went well, and what to fix, so you build real engineering habits instead of AI slop.

Features

• One chat for your agents. Use Codex, Claude Code or Gemini CLI from the PromptyQ sidebar. Chats are saved locally per project; reopen any chat from History and continue where you left off.
• Daily project reports. All chats in a project, from any agent, plus your own edits and test runs, become one report per day: a day score, a project score, and open items that carry over until fixed.
• Ten dimensions. Prompt craft, comprehension, verification & tests, security, design & scalability, code quality, debugging method, task decomposition, ownership and efficiency.
• Evidence, not vibes. About 65 deterministic checks (reading time vs. change size, files you opened, tests run, risky patterns in diffs, retry loops…) plus an optional daily AI review for the judgment calls rules can't make.
• Personal or team. Use it alone, or as a team where an admin adds members and sees only their team.

Requirements

• At least one coding-agent CLI installed and signed in: Codex CLI (also bundled with OpenAI's ChatGPT extension), Claude Code, or Gemini CLI.
• A free PromptyQ account (you can create it during sign-in).
• A trusted workspace: agents edit files, so PromptyQ doesn't run in Restricted Mode.

Getting started

1. After installing, the PromptyQ sidebar opens with a Sign in button (or click the owl in the activity bar).
2. Click Sign in. Your browser opens the PromptyQ sign-in page: sign in or create a free account, and you're sent back to VS Code signed in.
3. Click Turn on recording after reading what is collected (below). Nothing is captured until you agree.
4. Chat with your agent as usual. The icons under the prompt box open your dashboard, the admin page (personal accounts) and your account settings.

Members added by a team admin get a temporary password and set their own on the sign-in page the first time.

What is collected

Only after you run Enable Capture:

• your prompts and the agent's responses, and the agent's commands, tool calls and their results;
• a redacted, size-capped code diff of each agent turn (a git snapshot that never changes your files or index);
• compiler/lint diagnostics and test outcomes;
• which files you open or scroll, how many lines you type, terminal commands you run, and when the VS Code window gains or loses focus.

Common secret patterns (API keys, tokens, passwords) are redacted on your machine before upload. Data is buffered locally in a file only your user can read and uploaded over HTTPS to the server you signed in to. If your account belongs to a team, that team's admin can see your data; nobody outside your team or personal account can. You can reduce capture (promptyq.captureDiffs, promptyq.trackEditorActivity), turn it off (PromptyQ: Disable Capture), and delete all your data from the dashboard. Read the full privacy policy.

Commands

Settings