Naico AI Copilot Tracker

A VS Code extension that tracks and analyzes GitHub Copilot usage with detailed reports and metrics.

CONFIDENTIALITY NOTICE

NAICO ITS CONFIDENTIAL & PROPRIETARY

This software and its documentation contain proprietary information of NAICO ITS. It is provided under a license agreement containing restrictions on use and disclosure. Unauthorized reproduction, distribution, or disclosure is strictly prohibited. All rights reserved. © NAICO ITS.

Features

• Automatic Tracking: Always-on monitoring of Copilot-generated code
• Detailed Analytics: Daily and weekly usage statistics
• Language Breakdown: See which programming languages benefit most from Copilot
• Billing Savings: Calculate financial savings based on developer hourly rates
• Productivity Metrics: Estimate time saved with AI assistance
• Exportable Reports: Download CSV reports of your Copilot usage
• Team Performance: Compare productivity across your organization

Installation

1. Install from VS Code Marketplace:

   • Search for "NaicoITS".
   • Select Naico AI Copilot Tracker
   • Install.

2. Manual Installation via VSIX:

   • Download the latest VSIX file from the Releases Folder
   • In VS Code, go to Extensions view (Ctrl+Shift+X)
   • Click on the "..." at the top of the Extensions view
   • Select "Install from VSIX..."
   • Navigate to your downloaded VSIX file and select it

Setup

1. Enable the Extension:

   • After installation, the extension will be enabled by default
   • You can verify by checking the status bar for the Copilot tracking indicator

2. Configuration:

   • Open VS Code settings (Ctrl+,)
   • Search for "Naico AI Copilot"
   • Customize tracking preferences according to your needs
   • Available settings include:
     • naico.aiCopilotTracker.enabled: Enable/disable the extension
     • naico.aiCopilotTracker.trackingMode: Set the tracking level
     • naico.aiCopilotTracker.developerRate: Set your hourly rate for cost savings calculations

3. Usage:

   • The extension automatically tracks your GitHub Copilot interactions
   • View your statistics in the dedicated Copilot Tracker panel
   • Set your developer hourly rate to calculate financial savings
   • Export reports as needed through the panel options

Quick Start

1. Set your GitHub username via Command Palette: Set GitHub Username
2. Use Copilot as normal - tracking starts automatically
3. Open the Naico ITS sidebar to view stats
4. Access the report in multiple ways:
   • Click the graph icon in the status bar
   • Use the "View Report" button in the Naico ITS sidebar
   • Run the "View Copilot Tracking Report" command from the Command Palette

Dashboard Features

• Tracking Status: Shows active monitoring status
• Manual Refresh: Update stats on demand
• Multiple Views: Switch between Productivity, Daily, Weekly, and Billing Savings views
• Developer Rate Setting: Enter your hourly rate to calculate exact financial savings
• Export Options: Download usage reports by date or month
• Team Comparison: View how your usage compares to other team members

MongoDB Data Upload

For organizations that want to centralize Copilot usage tracking, the extension includes a MongoDB upload feature:

• One Entry Per User Per Day: Optimized data structure for organizational analytics
• Centralized Tracking: Upload usage statistics to a shared MongoDB database
• Team Insights: Compare productivity across developers on your team
• Simple Access: Upload data with one click using the Upload Data to MongoDB command
• Pre-Configured Connection: Uses a hardcoded connection string to Naico ITS MongoDB Atlas cluster
• Data Auto Sync feature: Syncing data with MongoDB every 10 mins

How to Use MongoDB Upload

1. Run the Upload Data to MongoDB command from the Command Palette.
2. Data for the current day will be uploaded to the database.
3. Check the output channel for success or error messages.

Organization Reports

After data is uploaded to MongoDB, you can access organization-level reports:

1. Open the Copilot Tracking Report
2. Navigate to the "Organization" tab
3. Enter the organization password when prompted
4. View consolidated metrics including:
   • Total time saved across your team
   • Lines of code generated by all developers
   • Cost savings based on developer rates
   • Active developer count and performance
5. Download monthly organization reports in CSV format for further analysis

Troubleshooting

If charts aren't displaying:

1. Click the Refresh button
2. Switch between tabs
3. For persistent issues, add these settings to your VS Code settings.json:

"naico-ai-copilot-tracker.chartColors": {
  "accentColor": "#0078D4",
  "borderColor": "#444444",
  "textColor": "#CCCCCC",
  "bgColor": "transparent"
}

Commands

• START Naico AI: Start tracking (if not already running)
• View Copilot Tracking Report: Open analytics dashboard
• Download Copilot Usage Report: Export data as CSV
• Download Monthly Report: Export monthly organization data
• Set GitHub Username: Configure your identity
• Upload Data to MongoDB: Share your stats with your organization

Data Privacy

• All data stored locally on your machine
• No data sent to external servers unless you explicitly use the MongoDB upload feature
• You control and can delete your data at any time
• Organization data is stored securely in MongoDB Atlas

How It Works

The extension tracks:

• Lines of code generated by GitHub Copilot
• Programming languages used
• Files modified with Copilot's assistance

Tracking Methodology

The extension uses a sophisticated heuristic approach to identify code generated by Copilot:

1. Copilot Activation Detection: Detects when Copilot is triggered to offer suggestions
2. Code Analysis: Analyzes inserted code for patterns typical of AI-generated content
3. Timing Analysis: Considers timing between suggestion and insertion
4. Content Filtering: Filters out empty lines and comment-only lines for accurate code counting
5. Structural Analysis: Requires code to have proper structure (functions, blocks, etc.)

This methodology provides a reasonable approximation of Copilot's contributions, though it may occasionally miss some suggestions or incorrectly attribute manually written code in rare cases.

Time and Cost Savings Calculation

Time savings are calculated using industry benchmarks:

• 30 keystrokes per line of code
• 0.4 minutes saved per Copilot-generated line (based on average typing speed and thinking time)

Financial savings are calculated by:

• Converting time saved to hours: (Lines of code × 0.4) ÷ 60
• Multiplying by your custom developer hourly rate
• Presenting daily, weekly, and monthly financial impact

Billing Savings Calculation

The Billing Savings tab helps you quantify the financial impact of using GitHub Copilot:

• Custom Developer Rate: Enter your hourly development rate
• Projected Savings: View calculated savings based on your actual Copilot usage
• ROI Analysis: See the return on investment for your Copilot subscription
• Cost Comparison: Compare manual coding costs vs. AI-assisted development
• Savings Reports: Download financial impact reports for sharing with management

Cost Savings Formula

The cost savings calculation uses the following formula:

1. Time saved (in hours) = (Lines of code × 0.4) ÷ 60

   • Where 0.4 is the estimated time in minutes saved per line
   • Divided by 60 to convert minutes to hours

2. Financial savings = Time saved × Developer hourly rate

   • Using your configured hourly rate or the default of $50/hour

For example, if Copilot generates 500 lines of code and your rate is $50/hour:

• Time saved = (500 × 0.4) ÷ 60 = 3.33 hours
• Financial savings = 3.33 × $50 = $166.50

How Copilot Code Detection Works

GitHub Copilot uses advanced machine learning models based on Transformer architecture to detect patterns in code and provide suggestions. Here's how the detection process works:

Pattern Recognition

• Copilot analyzes the surrounding code context, including variable names, function signatures, comments, and adjacent code logic
• It identifies programming patterns and structures that match its training data
• The AI recognizes coding conventions and common implementations for specific tasks

Contextual Understanding

• The tool examines the immediate context (current file) and broader project context
• It considers imports, dependencies, and referenced libraries
• Comments and documentation strings heavily influence the suggestions offered

Training and Knowledge Base

• Trained on billions of lines of public code from GitHub repositories
• Learns from various programming styles, patterns, and implementations
• Continuously improves through feedback mechanisms and model updates

Suggestion Generation

• Generates multiple possible completions ranked by relevance and likelihood
• Filters suggestions based on similarity to existing code in the repository
• Adapts to your coding style as you accept or reject suggestions

Limitations

• May occasionally suggest outdated or suboptimal code patterns
• The confidence of suggestions varies based on the uniqueness of the code context
• Can sometimes struggle with highly complex or unusual code structures

Copilot's detection capability improves over time as it learns from interactions and as the underlying models are enhanced.

More Information

• Visit our website
• Licensed under MIT License