AI Cockpit

Meet the AI Cockpit plugin, which offers a powerful integration with VS Code, providing simple and efficient authentication while ensuring direct access to the best LLMs (Large Language Models) available on the market. This feature allows developers to seamlessly leverage advanced artificial intelligence without the need for complex configurations. With this integration, it’s possible to take advantage of LLMs to accelerate development, generate more accurate code suggestions, and optimize workflows.

Additionally, the AI Cockpit stands out for its ability to access the context and backlog of the main work management tools, such as Jira, Azure DevOps, BusinessMap, and ServiceNow. This allows for a complete view of tasks and project progress directly within the development environment, offering a more integrated and efficient work experience. By connecting these tools to VS Code, the plugin makes it easier to navigate tasks, streamlining communication and managing the project lifecycle, making development more productive and organized.

AI Cockpit can handle complex software development tasks step-by-step. With tools that let him create & edit files, explore large projects, use the browser, and execute terminal commands (after you grant permission), he can assist you in ways that go beyond code completion or tech support. AI Cockpit can even use the Model Context Protocol (MCP) to create new tools and extend his own capabilities. While autonomous AI scripts traditionally run in sandboxed environments, this extension provides a human-in-the-loop GUI to approve every file change and terminal command, providing a safe and accessible way to explore the potential of agentic AI.

1. Enter your task and add images to convert mockups into functional apps or fix bugs with screenshots.
2. AI Cockpit starts by analyzing your file structure & source code ASTs, running regex searches, and reading relevant files to get up to speed in existing projects. By carefully managing what information is added to context, AI Cockpit can provide valuable assistance even for large, complex projects without overwhelming the context window.
3. Once AI Cockpit has the information he needs, he can:
   • Create and edit files + monitor linter/compiler errors along the way, letting him proactively fix issues like missing imports and syntax errors on his own.
   • Execute commands directly in your terminal and monitor their output as he works, letting him e.g., react to dev server issues after editing a file.
   • For web development tasks, AI Cockpit can launch the site in a headless browser, click, type, scroll, and capture screenshots + console logs, allowing him to fix runtime errors and visual bugs.
4. When a task is completed, AI Cockpit will present the result to you with a terminal command like open -a "Google Chrome" index.html, which you run with a click of a button.

[!TIP] Use the CMD/CTRL + Shift + P shortcut to open the command palette and type "AI Cockpit: Open In New Tab" to open the extension as a tab in your editor. This lets you use AI Cockpit side-by-side with your file explorer, and see how he changes your workspace more clearly.

Use any API and Model

AI Cockpit supports API providers like OpenRouter, Anthropic, OpenAI, Google Gemini, AWS Bedrock, Azure, and GCP Vertex. You can also configure any OpenAI compatible API, or use a local model through LM Studio/Ollama. If you're using OpenRouter, the extension fetches their latest model list, allowing you to use the newest models as soon as they're available.

The extension also keeps track of total tokens and API usage cost for the entire task loop and individual requests, keeping you informed of spend every step of the way.

Run Commands in Terminal

Thanks to the new shell integration updates in VSCode v1.93, AI Cockpit can execute commands directly in your terminal and receive the output. This allows him to perform a wide range of tasks, from installing packages and running build scripts to deploying applications, managing databases, and executing tests, all while adapting to your dev environment & toolchain to get the job done right.

For long running processes like dev servers, use the "Proceed While Running" button to let AI Cockpit continue in the task while the command runs in the background. As AI Cockpit works he’ll be notified of any new terminal output along the way, letting him react to issues that may come up, such as compile-time errors when editing files.

Create and Edit Files

AI Cockpit can create and edit files directly in your editor, presenting you a diff view of the changes. You can edit or revert AI Cockpit's changes directly in the diff view editor, or provide feedback in chat until you're satisfied with the result. AI Cockpit also monitors linter/compiler errors (missing imports, syntax errors, etc.) so he can fix issues that come up along the way on his own.

All changes made by AI Cockpit are recorded in your file's Timeline, providing an easy way to track and revert modifications if needed.

Use the Browser

With Claude 3.5 Sonnet's new Computer Use capability, AI Cockpit can launch a browser, click elements, type text, and scroll, capturing screenshots and console logs at each step. This allows for interactive debugging, end-to-end testing, and even general web use! This gives him autonomy to fixing visual bugs and runtime issues without you needing to handhold and copy-pasting error logs yourself.

Try asking AI Cockpit to "test the app", and watch as he runs a command like npm run dev, launches your locally running dev server in a browser, and performs a series of tests to confirm that everything works. See a demo here.

"add a tool that..."

Thanks to the Model Context Protocol, AI Cockpit can extend his capabilities through custom tools. While you can use community-made servers, AI Cockpit can instead create and install tools tailored to your specific workflow. Just ask AI Cockpit to "add a tool" and he will handle everything, from creating a new MCP server to installing it into the extension. These custom tools then become part of AI Cockpit's toolkit, ready to use in future tasks.

• "add a tool that fetches Jira tickets": Retrieve ticket ACs and put AI Cockpit to work
• "add a tool that manages AWS EC2s": Check server metrics and scale instances up or down
• "add a tool that pulls the latest PagerDuty incidents": Fetch details and ask AI Cockpit to fix bugs

Add Context

@url: Paste in a URL for the extension to fetch and convert to markdown, useful when you want to give AI Cockpit the latest docs

@problems: Add workspace errors and warnings ('Problems' panel) for AI Cockpit to fix

@file: Adds a file's contents so you don't have to waste API requests approving read file (+ type to search files)

@folder: Adds folder's files all at once to speed up your workflow even more

Checkpoints: Compare and Restore

As AI Cockpit works through a task, the extension takes a snapshot of your workspace at each step. You can use the 'Compare' button to see a diff between the snapshot and your current workspace, and the 'Restore' button to roll back to that point.

For example, when working with a local web server, you can use 'Restore Workspace Only' to quickly test different versions of your app, then use 'Restore Task and Workspace' when you find the version you want to continue building from. This lets you safely explore different approaches without losing progress.

AI Cockpit Enhancements to Cline:

• AI Cockpit authentication provider
• AI Cockpit streamlined context sharing
• AI Cockpit authorized LLMs based on organization
  
  

License

This project is based on Cline, which is licensed under the Apache 2.0 License.