TZ Copilot Telemetry Collector

Track and analyze your GitHub Copilot usage metrics directly in Visual Studio Code. This extension provides comprehensive analytics and insights into your AI-assisted coding activities.

Features

• 📊 Real-time Metrics Dashboard: View detailed analytics of your Copilot usage including completions, chat generations, lines and characters generated
• 📁 Workspace Tracking: Monitor metrics per workspace with automatic workspace detection
• 💾 Export Functionality: Download metrics data as JSON for further analysis
• 🔄 Reset Capabilities: Clear metrics data when needed
• 📡 Telemetry Configuration: Send metrics to your organization's telemetry server for centralized analytics and reporting
• 🔐 GitHub Integration: Secure authentication with GitHub for user identification
• 📋 Activity Logs: View detailed logs of extension activities
• 📈 Status Monitoring: Real-time status indicator in the status bar

Commands

The extension provides the following commands available via the Command Palette (Ctrl+Shift+P):

• TZ Copilot Telemetry Collector: Preview Metrics - Opens a detailed dashboard showing all collected metrics
• TZ Copilot Telemetry Collector: Download Metrics - Exports metrics data as a JSON file
• TZ Copilot Telemetry Collector: Reset Metrics - Clears all collected metrics data
• TZ Copilot Telemetry Collector: Status - Shows current telemetry status and configuration
• TZ Copilot Telemetry Collector: Set Telemetry Config - Configure telemetry server and authentication
• TZ Copilot Telemetry Collector: Clear Telemetry Config - Remove telemetry configuration
• TZ Copilot Telemetry Collector: Show Logs - Display extension activity logs

Requirements

• Visual Studio Code version 1.101.0 or higher
• GitHub account with active Copilot license (for authentication features)
• Node.js runtime (for telemetry server communication)
• Access to your organization's telemetry server (for centralized data collection)(Optional)

Extension Settings

This extension contributes the following settings:

• tzCopilotTelemetryCollector.telemetryEnabled: Enable/disable telemetry data collection (default: false)
• tzCopilotTelemetryCollector.telemetryServerUrl: URL endpoint for your organization's telemetry server (e.g., https://metrics.yourcompany.com/api/copilot)
• tzCopilotTelemetryCollector.telemetryApiKey: API key for authenticating with your telemetry server
• tzCopilotTelemetryCollector.projectName: Project name for organizing telemetry data within your organization
• tzCopilotTelemetryCollector.githubUsername: GitHub username with Copilot license for user identification

Getting Started

1. Install the extension from the VS Code marketplace
2. Open a workspace in VS Code
3. Use Ctrl+Shift+P and run "TZ Copilot Telemetry Collector: Preview Metrics" to see your dashboard
4. (Optional) Configure telemetry by running "TZ Copilot Telemetry Collector: Set Telemetry Config"

Telemetry Setup (Optional)

Why Use Telemetry?

Telemetry allows your organization to:

• Centralize Analytics: Collect Copilot usage data from all developers in one place
• Track ROI: Measure the impact of GitHub Copilot investment across teams
• Identify Patterns: Understand which projects and developers benefit most from AI assistance
• Compliance: Meet organizational requirements for tracking AI tool usage
• Reporting: Generate comprehensive reports for management and stakeholders

Setting Up Your Telemetry Server (Optional)

The extension sends metrics to your organization's telemetry server. You'll need:

1. Server URL

Your organization's telemetry endpoint URL, typically in the format:

https://metrics.yourcompany.com/api/copilot

This endpoint should accept POST requests with JSON payloads containing metrics data.

2. API Key

A secure authentication token provided by your organization's IT team or telemetry server administrator. This ensures only authorized users can send data to your telemetry system.

3. Project Name

A identifier for your project or team, used to organize and filter metrics data within your organization's telemetry system.

Configuration Steps

1. Get Credentials: Contact your IT team or telemetry server administrator to obtain:

   • Telemetry server URL
   • API key for authentication
   • Project name (if standardized)

2. Configure Extension:

   • Open Command Palette (Ctrl+Shift+P)
   • Run "TZ Copilot Telemetry Collector: Set Telemetry Config"
   • Enter your telemetry server URL
   • Enter your API key
   • Enter your project name
   • Confirm to enable telemetry

3. Verify Setup:

   • Run "TZ Copilot Telemetry Collector: Status" to check configuration
   • Look for "Telemetry: Enabled" in the status display

Expected Server API Format

Your telemetry server should accept POST requests with this JSON structure:

{
  "username": "github-username",
  "project": "project-name",
  "workspace": "workspace-name",
  "metrics": {
    "totalCompletions": 123,
    "totalChatGenerations": 45,
    "totalLinesGenerated": 678,
    "totalCharactersGenerated": 9012,
    "filesModified": ["file1.ts", "file2.js"]
  },
  "timestamp": "2024-01-01T12:00:00Z"
}

Data Collection

The extension tracks the following metrics:

• Workspace Name: Current root directory name
• Total Completions: Number of code completions accepted
• Total Chat Generations: Number of chat interactions with Copilot
• Total Lines Generated: Total lines of code generated
• Total Characters Generated: Total characters generated by Copilot
• Files Modified: List of files where Copilot assistance was used

Privacy & Security

• All metrics data is stored locally in VS Code's global storage
• GitHub authentication uses VS Code's built-in authentication provider
• Telemetry server communication is optional and user-controlled
• API keys are stored securely in VS Code settings and never logged
• Data transmission to telemetry server uses HTTPS encryption
• No data is transmitted without explicit user configuration and consent

Known Issues

• Metrics tracking requires active file editing to detect Copilot usage
• GitHub authentication requires internet connectivity
• Telemetry server must support the expected API format

Calling out known issues can help limit users opening duplicate issues against your extension.

Release Notes

Users appreciate release notes as you update your extension.

1.0.0

This release contains the main functionalities of collection copilot metrics.

Following extension guidelines

Ensure that you've read through the extensions guidelines and follow the best practices for creating your extension.

• Extension Guidelines

Working with Markdown

You can author your README using Visual Studio Code. Here are some useful editor keyboard shortcuts:

• Split the editor (Cmd+\ on macOS or Ctrl+\ on Windows and Linux).
• Toggle preview (Shift+Cmd+V on macOS or Shift+Ctrl+V on Windows and Linux).
• Press Ctrl+Space (Windows, Linux, macOS) to see a list of Markdown snippets.

For more information

• Visual Studio Code's Markdown Support
• Markdown Syntax Reference