CogniVision

Meet CogniVision, an AI assistant specialized in Augmented Reality (AR) development and computer vision tasks.

Thanks to advanced agentic capabilities, CogniVision can handle complex AR-related software development tasks step-by-step. Whether you are building AR experiences, integrating computer vision models, or refining AR interfaces, CogniVision acts as your dedicated AR development partner. While autonomous AI scripts traditionally run in sandboxed environments, this extension provides a human-in-the-loop GUI to approve every file change and terminal command, providing a safe and accessible way to explore the potential of agentic AI.

1. Enter your task and add images to convert mockups into functional AR applications or fix tracking and rendering bugs with screenshots.
2. CogniVision starts by analyzing your file structure & source code ASTs, running regex searches, and reading relevant files to get up to speed in existing AR projects.
3. Once CogniVision has the information he needs, he can:
   • Create and edit files + monitor linter/compiler errors along the way, proactively fixing AR rendering issues and syntax errors on his own.
   • Execute commands directly in your terminal and monitor their output as he works, letting him e.g., react to dev server or simulator issues after editing a file.
   • For AR development tasks, CogniVision can debug spatial computing logic, click, type, scroll, and capture logs, allowing him to fix runtime errors and visual bugs in Augmented Reality environments.
4. When a task is completed, CogniVision will present the result to you.

[!TIP] Follow to open CogniVision on the right side of your editor. This lets you use CogniVision side-by-side with your file explorer, and see how he changes your workspace more clearly.

Run Commands in Terminal

Thanks to the new shell integration updates in VSCode v1.93, CogniVision can execute commands directly in your terminal and receive the output. This allows him to perform a wide range of tasks, from installing packages and running build scripts to deploying applications, managing databases, and executing tests, all while adapting to your dev environment & toolchain to get the job done right.

For long running processes like dev servers, use the "Proceed While Running" button to let CogniVision continue in the task while the command runs in the background. As CogniVision works he’ll be notified of any new terminal output along the way, letting him react to issues that may come up, such as compile-time errors when editing files.

Create and Edit Files

CogniVision can create and edit files directly in your editor, presenting you a diff view of the changes. You can edit or revert CogniVision's changes directly in the diff view editor, or provide feedback in chat until you're satisfied with the result. CogniVision also monitors linter/compiler errors (missing imports, syntax errors, etc.) so he can fix issues that come up along the way on his own.

All changes made by CogniVision are recorded in your file's Timeline, providing an easy way to track and revert modifications if needed.

Use the Browser

With Claude Sonnet's new Computer Use capability, CogniVision can launch a browser, click elements, type text, and scroll, capturing screenshots and console logs at each step. This allows for interactive debugging, end-to-end testing, and even general web use! This gives him autonomy to fixing visual bugs and runtime issues without you needing to handhold and copy-pasting error logs yourself.

Try asking CogniVision to "test the app", and watch as he runs a command like npm run dev, launches your locally running dev server in a browser, and performs a series of tests to confirm that everything works. See a demo here.

"add a tool that..."

Thanks to the Model Context Protocol, CogniVision can extend his capabilities through custom tools. While you can use community-made servers, CogniVision can instead create and install tools tailored to your specific workflow. Just ask CogniVision to "add a tool" and he will handle everything, from creating a new MCP server to installing it into the extension. These custom tools then become part of CogniVision's toolkit, ready to use in future tasks.

• "add a tool that fetches Jira tickets": Retrieve ticket ACs and put CogniVision to work
• "add a tool that manages AWS EC2s": Check server metrics and scale instances up or down
• "add a tool that pulls the latest PagerDuty incidents": Fetch details and ask CogniVision to fix bugs

Add Context

@url: Paste in a URL for the extension to fetch and convert to markdown, useful when you want to give CogniVision the latest docs

@problems: Add workspace errors and warnings ('Problems' panel) for CogniVision to fix

@file: Adds a file's contents so you don't have to waste API requests approving read file (+ type to search files)

@folder: Adds folder's files all at once to speed up your workflow even more

Checkpoints: Compare and Restore

As CogniVision works through a task, the extension takes a snapshot of your workspace at each step. You can use the 'Compare' button to see a diff between the snapshot and your current workspace, and the 'Restore' button to roll back to that point.

For example, when working with a local web server, you can use 'Restore Workspace Only' to quickly test different versions of your app, then use 'Restore Task and Workspace' when you find the version you want to continue building from. This lets you safely explore different approaches without losing progress.

Contributing

To contribute to the project, start with our Contributing Guide to learn the basics. You can also join our to chat with other contributors in the #contributors channel. If you're looking for full-time work, check out our open positions on our !

Enterprise

Get the same CogniVision experience with enterprise-grade controls: SSO (SAML/OIDC), global policies and configuration, observability with audit trails, private networking (VPC/private link), and self-hosted or on-prem deployments, and enterprise support. Learn more at our or .

License

Apache 2.0 © 2026 CogniVision Bot Inc.