CogniVision

Your AI Partner for Augmented Reality Development

Overview

Building Augmented Reality experiences is hard. You're juggling spatial anchors, tracking algorithms, shader code, device-specific APIs, and real-time rendering pipelines — all at once. Most AI assistants give you generic code that breaks the moment it touches an AR runtime.

CogniVision is different. It's an AI coding agent built specifically for AR and spatial computing development, embedded directly in Visual Studio Code. Describe the AR experience you want to create — a markerless anchor, a face filter, a WebXR scene, a LiDAR-based occlusion mesh — and CogniVision writes accurate, production-ready code, runs it through your terminal, reads the output, and iterates until the result is right.

Every file it edits, every command it runs, every decision it makes is visible to you in real time. You stay in control. You approve changes before they land.

What CogniVision Can Build

ARKit / RealityKit scenes for iOS (Swift, Objective-C)
ARCore / Sceneform experiences for Android (Kotlin, Java)
WebXR / Three.js / A-Frame web-based AR
Unity AR Foundation cross-platform scenes (C#)
Computer vision pipelines — object detection, segmentation, pose estimation
Shader and material code for spatial rendering
Spatial UI — world-space panels, gaze interaction, hand tracking input

Key Features

🏗️ AR-First Code Generation

CogniVision knows the APIs, lifecycle hooks, and spatial math behind real AR frameworks. When you ask it to implement plane detection, it doesn't give you a stub — it gives you working ARPlaneAnchor delegates, session configuration, and a sceneView setup wired together correctly.

🔍 Deep Project Understanding

Before writing anything, CogniVision reads your project: file structure, scene graphs, existing shaders, manifest files, and build configs. It understands what you already have, so it builds on top of your work — not over it.

🐛 Spatial Debugging

AR apps fail in ways general-purpose debuggers don't anticipate: anchor drift, tracking loss, occlusion gaps, incorrect world transforms, or frame-rate drops in render passes. CogniVision runs your app in the terminal, reads crash logs and console output, and proposes targeted fixes based on what it sees.

👁️ Real-Time Transparency

Every action CogniVision takes is shown as it happens in the chat panel. You can see which files it's reading, what code it's writing, and what commands it's running — all before they're finalized. Nothing happens without your awareness.

✅ Human-in-the-Loop by Design

Every file write and terminal command surfaces a confirmation step. You can accept, modify, or reject any change. Workspace snapshots are taken at each step, so you can roll back to any previous state at any time.

🧩 Extensible Tool System

CogniVision supports the Model Context Protocol (MCP), allowing you to connect external data sources — device sensor feeds, AR cloud services, spatial databases — directly into the agent's toolchain.

How It Works

You describe an AR goal in plain language
        ↓
CogniVision reads your project structure and relevant files
        ↓
It plans an implementation and shows you the approach
        ↓
It writes code, runs commands, reads output, and iterates
        ↓
Every change is shown to you before it's applied
        ↓
Task complete — you review the result

Installation

From the VS Code Marketplace

Open the Extensions panel (Ctrl+Shift+X / Cmd+Shift+X)
Search for CogniVision
Click Install

Manual Install

Download the latest .vsix from the Releases page
Open VS Code → Extensions panel → ··· menu → Install from VSIX…
Select the downloaded file

Usage

Starting an AR Task

Open the CogniVision panel from the Activity Bar (the AR eye icon). Type what you want to build or fix in the chat input and press Enter.

Examples:

"Add plane detection to my ARKit scene and place a 3D object when the user taps"
"My AR anchor is drifting every 30 seconds on Android — investigate and fix"
"Create a WebXR scene with hand tracking input using Three.js"
"Write a metal shader for real-time environment occlusion in RealityKit"

Providing Context

@file — Include a specific source file in the conversation
@folder — Include all files in a folder (e.g., your Shaders or Scenes directory)
@url — Fetch documentation from a URL (ARKit docs, WebXR spec, etc.)
@problems — Attach the current list of build errors and warnings

Reviewing Changes

CogniVision shows a diff for every file it modifies. You can:

Accept the change as written
Edit the diff directly in the editor
Reject and provide feedback to try again

Checkpoints and Rollback

CogniVision snapshots your workspace before each significant change. Use the Compare button to see what changed, or Restore to roll back to any previous checkpoint — useful when testing different approaches to a spatial rendering problem.

Unique Differentiators

Feature	What It Means for AR Developers
AR-domain knowledge	Generates accurate ARKit, ARCore, WebXR, and Unity AR Foundation code out of the box
Framework-aware debugging	Reads AR runtime logs and suggests fixes based on spatial computing failure patterns
Persistent project context	Remembers your workspace conversation across VS Code sessions — no need to re-explain your scene setup
Real-time action streaming	See every tool call as it executes in the chat, not just a final result
Scene graph awareness	Understands node hierarchies, anchor relationships, and transform chains before making changes

Contributing

Pull requests and issues are welcome. See CONTRIBUTING.md for guidelines.

AR Dev Assistant