Copilot Radar: AI Observability & Cost Monitor

Real-time AI telemetry, context saturation tracking, cost burn, and anti-thrashing heuristics for GitHub Copilot.

1. Description

Copilot Radar (published by codescope-ai) is an ambient AI Coding Observability & Cost Flight Recorder running directly alongside GitHub Copilot across any workspace or repository.

Unlike AI coding assistants that generate code, Copilot Radar acts as the developer's observability flight recorder and anti-thrashing coach:

• 100% Native Copilot Experience: No @ mentions or commands required. Chat with GitHub Copilot normally and code as you always do.
• Zero-Touch Passive Observability: Copilot Radar quietly observes session context saturation, token burn, cost accumulation, and anti-thrashing heuristics in the background.
• Flight Recorder & Anti-Thrashing Coach: Analyzes 128k context window saturation %, token burn, live USD cost, retry loops, file-level rework, and code thrashing.

2. Why Copilot Radar is Useful

As AI coding assistants become standard in software engineering, developers face critical invisible bottlenecks that standard IDEs fail to measure:

The 5 Problems Copilot Radar Solves:

3. Architecture & Codebase Explanation

Copilot Radar employs a decoupled, high-performance architecture combining a Python AI Engine with a Thin VS Code Client:

flowchart TD
    subgraph IDE ["VS Code Environment"]
        Copilot["GitHub Copilot (Chat & Inline Ghost Text)"]
        Panel["Copilot Radar Bottom Panel (Webview View)"]
        StatusBar["Status Bar Item ($(radio-tower) Radar 100)"]
    end

    subgraph VSCodeExt ["VS Code Extension Host (TypeScript)"]
        PassiveWatcher["Passive Copilot Store & Editor Observer"]
        ExtensionCore["extension.ts (State Sync & Diagnostics)"]
    end

    subgraph LocalStorage ["Local Persistent Workspace Store"]
        TelemetryJSON[".codescope/telemetry.json"]
    end

    subgraph PythonCore ["Python AI Observability Core (codescope/)"]
        TokenEstimator["token_estimator.py (Pricing & Saturation)"]
        ThrashingDetector["thrashing_detector.py (Friction & Undos)"]
        SecretRedactor["secret_redactor.py (Client-side Scrubbing)"]
        SessionEngine["session_engine.py (Convergence Velocity)"]
        CopilotWatcher["copilot_watcher.py (Passive Store Reader)"]
        CLI["cli.py (Terminal Dashboard & Status)"]
        Daemon["server/daemon.py (Local HTTP Server :4545)"]
    end

    Copilot -.->|Writes chat sessions & completions| PassiveWatcher
    PassiveWatcher --> ExtensionCore
    ExtensionCore -->|Renders live metrics| Panel
    ExtensionCore -->|Updates score| StatusBar
    ExtensionCore <-->|Atomic sync| TelemetryJSON
    TelemetryJSON <-->|Read / Write / Analyze| PythonCore
    PythonCore -->|Interactive terminal CLI| CLI

Module Breakdown

Python AI Core (codescope/)

• codescope/core/models.py: Strongly-typed dataclasses (TelemetryRecord, FileThrashingStats, TokenMetrics, TimelineEvent).
• codescope/core/token_estimator.py: Model pricing matrix (Claude 3.5 Sonnet, GPT-4o, Codex), subword tokenization, and context window saturation calculations.
• codescope/core/thrashing_detector.py: Sliding-window tracking of completions, retries, rapid undos (<60s after insertion), and classification (converged, friction, thrashing).
• codescope/core/secret_redactor.py: Fast regex-based credential and secret redaction ([REDACTED]).
• codescope/core/prompt_evaluator.py: AI Efficiency Score (AES) evaluation across specificity, context grounding, outcome clarity, and constraints.
• codescope/core/session_engine.py: Session lifecycle management, convergence velocity, rework index, and .codescope/telemetry.json state synchronization.
• codescope/core/copilot_watcher.py: Passive JSON store watcher that reads VS Code active Copilot chat conversations without requiring @ prefix mentions.
• codescope/cli.py: Terminal interface for status, monitor, export, and daemon.
• codescope/server/daemon.py: Zero-dependency local HTTP API daemon on port 4545 (/api/session, /api/status).

VS Code Extension (packages/codescope-vscode/)

• packages/codescope-vscode/src/extension.ts:
  • Context Saturation & Token Burn UI: Progress bar visualizing context saturation %, input/output token counts, and live cost calculation.
  • File Rework & Thrashing Heatmap UI: Table rendering per-file completion counts, retries, undos, and friction scores.
  • Anti-Thrashing AI Coach UI: Dynamic guidance advising developers on context grounding and task decomposition.
  • Integrated Action Bar: Direct buttons for [Export Telemetry Report], [Run Python Diagnostics], and [Load Demo Scenario].
  • Zero-Touch Observability: Passively watches local Copilot chat stores and editor events with zero chat disruption.

4. Installation & Usage Guide

A. Installing from the Visual Studio Marketplace (Recommended)

Copilot Radar is published and available directly on the Visual Studio Marketplace.

Method 1: Inside VS Code (One Click)

1. Open VS Code.
2. Press Cmd + Shift + X (or Ctrl + Shift + X on Windows/Linux) to open the Extensions view.
3. Search for Copilot Radar (or codescope-ai.codescope-vscode).
4. Click Install.

Method 2: Via Terminal

code --install-extension codescope-ai.codescope-vscode

Method 3: Manual VSIX Installation (Air-Gapped / Local Development)

If you are developing locally or in an isolated environment:

1. In VS Code, open the Command Palette (Cmd + Shift + P or Ctrl + Shift + P).
2. Type Extensions: Install from VSIX... and select codescope-vscode-1.0.6.vsix.

B. Using the Copilot Radar Observability Panel

1. Open the bottom panel in VS Code (where Terminal / Output are located) and click the Copilot Radar tab.
2. Code normally using GitHub Copilot (chat or inline completions). The panel updates automatically with zero configuration:
   • Today's Usage (Daily Aggregate): Cumulative AI sessions, total turns, accepted rate, retry rate, token counts, and estimated USD spend across your entire workday.
   • Active Task Session: Real-time AI Efficiency Score (AES / 100), active turns, convergence %, task cost, and context saturation.
   • Context Saturation Bar: Tracks active task token consumption against the 128k context limit.
   • File Thrashing Heatmap: Highlights files with high churn, retries, or repeated undo loops.
   • Anti-Thrashing AI Coach: Provides proactive recommendations to prevent model degradation and prompt drift.
   • AI Session Timeline: Detailed chronological feed with timestamps, model engines, file references, tokens in/out, and outcome status.
3. + New Task: Click [+ New Task] or run codescope.newTask whenever you switch tasks. This archives the active session into .codescope/sessions/<sessionId>.json, starts a fresh task session counter at 0, preserves your daily TODAY metrics, and starts a fresh Copilot chat.
4. 📋 Copy PR Summary: Click [📋 Copy PR Summary] or run codescope.copyPrSummary to generate a formatted markdown summary of your AI observability metrics to paste into GitHub Pull Request descriptions.

C. Running in the Terminal (Python CLI)

You can inspect and monitor telemetry directly from any terminal window:

1. View Session Telemetry & Heatmap

python3 -m codescope status

2. Live Terminal Monitor (Auto-refreshing)

python3 -m codescope monitor

3. Export Telemetry

python3 -m codescope export

4. Run Local HTTP Telemetry Daemon

python3 -m codescope daemon

D. Build & Verification Commands

# Compile TypeScript Core & VS Code Extension
npm run build
npm run build:extension

# Run Comprehensive Unified Test Suite (TypeScript + Python)
npm test

# Run Python Tests Only
npm run test:py

5. Configuration (codescope.yaml)

Customize telemetry behavior by editing codescope.yaml in your project root:

codescope:
  enabled: true
  mode: local

telemetry:
  usage: true
  acceptance: true
  prompt_content: false # Privacy by default: prompt content is never egressed
  source_code: false    # Source code is NEVER collected or uploaded
  token_estimation: true

privacy:
  redact_secrets: true
  retention_days: 30

6. License

Apache-2.0 License.