VSCode Copilot Metrics

Overview 🚀

VSCode Copilot Metrics is an extension that tracks usage statistics of GitHub Copilot within your Visual Studio Code environment. It provides insights into completion counts, generated code character counts (via tab key presses), and chat codeblock generation, giving users a clear understanding of their AI-assisted coding productivity.

While the extension's name references "Copilot," it remains AI-agnostic for models that offer inlay completions. However, for chat codeblock generation, the extension currently tracks only GitHub Copilot interactions.

Status Bar Integration 📊

Each workspace maintains its own set of statistics, accessible via the status bar or command palette. The preview panel presents the following key metrics:

• Completions: Total number of AI-generated completions
• Lines: Total number of code lines generated
• Chars: Total number of code characters generated
• Chat: Total number of chat codeblocks generated
• Files: Breakdown of completions and character counts by file

Workspace-specific statistics persist even after closing the workspace or editor, ensuring continuity across sessions. Data is stored within the user's home directory, organized by workspace.

Quick Access Menu 📋

A convenient quick menu, accessible via the status bar item, provides several functionalities:

• Statistics Preview: Open a webview panel displaying a formatted preview of the JSON file containing statistics
• Statistics Download: Export collected statistics as a JSON file
• Reset Statistics: Clear all tracked statistics
• Set Telemetry Config: Configure the telemetry server address, API key, username, and project name
• Enable/Disable Telemetry: Toggle telemetry on or off
• Clear Telemetry Config: Remove all telemetry configurations
• View Extension Logs: Access extension logs for troubleshooting or reference

Exported JSON Structure

{
  "workspace": "vscode-copilot-metrics",
  "total_completions_count": 455,
  "total_lines_generated": 645,
  "total_chars_generated": 10769,
  "total_chat_generations": 8,
  "files": [
    {
      "file_path": "/your/workspace/path/file1.ts",
      "language_id": "typescript",
      "completions_count": 1,
      "generated_chars": 10
    }
  ]
}

Telemetry 📡

The extension provides integration with a telemetry server for comprehensive project usage metrics analysis.

Configuration

Users can configure the following details in their session:

• Telemetry server address
• API key (sent as x-api-key header)
• Username
• Project name

The server address, API key, and username are saved in the global session, while the project name is saved in the workspace session.

Customizable Endpoints

You can personalize the following endpoints in the extension settings:

1. Metrics endpoint: for sending metrics data
2. Projects endpoint: for retrieving the list of projects to display in the quickpick (optional)

Username Configuration

The extension will ask to authenticate with your GitHub account. The list of emails will be shown, and you can select the one you want to use.

Project Name Configuration

Configure the username and project name in two ways:

1. Enter the value directly in the input field
2. Configure endpoints in the extension settings to activate a remote quickpick, fetching the list of projects from your backend

For remote quickpick, the request should follow this structure:

[
  {
    "label": "string",
    "description": "string" // optional
  }
]

Remote Collection

Once configured, metrics are dispatched individually for each accepted suggestion. This facilitates remote collection of fine-grained data, allowing integration into preferred analytics platforms.

Telemetry payload structure:

{
  username: string;
  project_name: string;
  total_lines_generated: number;
  total_chars_generated: number;
  language_id: string;
  editor: "vscode";
  plugin_version: string;
}

Requirements 📦

No specific prerequisites are necessary. The extension seamlessly integrates with any AI model offering inlay completions within Visual Studio Code.

Author 🧑‍💻

Flavio Del Grosso | @flaviodelgrosso